Controlling Remote Electronic Device with Wearable Electronic Device

ABSTRACT

In one embodiment, a method includes displaying functions associated with a remote device at a wearable computing device, wherein the remote device is paired with the wearable computing device, receiving a user input from a user at the wearable computing device, determining intended operations of one or more of the functions by the user based on the user input, sending instructions for executing the intended operations of the one or more of the functions associated with the remote device to the remote device, and displaying execution results of the intended operations of the one or more of the functions associated with the remote device at the wearable computing device.

PRIORITY

This application is a continuation under 35 U.S.C. § 120 of U.S. patentapplication Ser. No. 14/015,947, filed 30 Aug. 2013, which claims thebenefit, under 35 U.S.C. § 119(e), of U.S. Provisional PatentApplication No. 61/728,765, filed 20 Nov. 2012, U.S. Provisional PatentApplication No. 61/728,770, filed 20 Nov. 2012, U.S. Provisional PatentApplication No. 61/773,803, filed 6 Mar. 2013, U.S. Provisional PatentApplication No. 61/728,773, filed 20 Nov. 2012, U.S. Provisional PatentApplication No. 61/773,813, filed 7 Mar. 2013, U.S. Provisional PatentApplication No. 61/773,815, filed 7 Mar. 2013, U.S. Provisional PatentApplication No. 61/773,817, filed 7 Mar. 2013, U.S. Provisional PatentApplication No. 61/775,688, filed 11 Mar. 2013, U.S. Provisional PatentApplication No. 61/775,687, filed 11 Mar. 2013, and U.S. ProvisionalPatent Application No. 61/775,686, filed 11 Mar. 2013, which are allincorporated herein by reference.

TECHNICAL FIELD

This disclosure generally relates to a wearable electronic device.

BACKGROUND

Mobile electronic devices provide a user with access to computingcapabilities even as the user moves about various locations. Examples ofmobile electronic devices include mobile phones, media players, laptops,tablets, PDAs, or hybrid devices that include functionality of multipledevices of this type.

Mobile electronic devices may be part of a communication network such asa local area network, wide area network, cellular network, the Internet,or any other suitable network. A mobile electronic device may use acommunication network to communicate with other electronic devices, forexample, to access remotely-stored data, access remote processing power,access remote displays, provide locally-stored data, provide localprocessing power, or provide access to local displays. For example,networks may provide communication paths and links to servers, which mayhost applications, content, and services that may be accessed orutilized by users via mobile electronic devices. The content may includetext, video data, audio data, user settings or other types of data.Networks may use any suitable communication protocol or technology tofacilitate communication between mobile electronic devices, such as, forexample, BLUETOOTH, IEEE WI-FI (802.11a/b/g/n/ac), or TCP/IP.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example embodiment of an wearable electronicdevice.

FIG. 2 illustrates an example stack-up of a device.

FIGS. 3A-3E illustrate example form factors of a device.

FIG. 4A illustrates an example cross-section of a device body.

FIGS. 4B-C illustrate example connections between components of adevice.

FIGS. 5A-5F illustrate example displays of a device.

FIGS. 6A-C illustrate example cross-sectional views of a device display.

FIGS. 7A-7D illustrate example outer elements about a device body.

FIGS. 8A-8C illustrate example outer elements about a device body.

FIG. 9 illustrates an example sealing ring of a device.

FIG. 10 illustrates an example retention ring of a device.

FIG. 11 illustrates various example embodiments for wearing a device.

FIGS. 12A-12B illustrate a band attached to a body of a device.

FIGS. 13A-13I illustrate example embodiments for fastening or affixing aband of a device.

FIGS. 14A-D illustrate example camera placements on a device.

FIG. 15 illustrates an example device with a band and optical sensor.

FIG. 16 illustrates an example viewing triangle including a user, adevice, and an object.

FIG. 17 illustrates an example angle of view for an optical sensor of adevice.

FIGS. 18A-18B illustrate example optical sensors of a device.

FIG. 19 illustrates an example sensor detection system of a device.

FIGS. 20A-20C illustrate example chargers operable with a device.

FIGS. 21A-21B illustrate example chargers operable with a device.

FIGS. 22A-22B illustrate example charging units operable with a device.

FIG. 23 illustrates an example charging scheme for a charging unitoperable with a device.

FIG. 24 illustrates an example charging scheme for a charging unitoperable with a device.

FIGS. 25A-25E illustrate example embodiments of energy storage andcharging in a device and a charging unit.

FIG. 26 illustrates an example charging unit architecture.

FIGS. 27-92 illustrate example gestures for use with a device.

FIGS. 93A-93B illustrate example user inputs to a device.

FIGS. 94A-94C illustrate example user inputs to a device.

FIGS. 95A-95D illustrate example user touch input to a device.

FIGS. 96A-96B illustrate example graphical user interface models of adevice.

FIG. 97 illustrates an example graphical user interface model of adevice.

FIGS. 98A-98G illustrate example graphical user interface models of adevice.

FIG. 99 illustrates an example graphical user interface model of adevice.

FIGS. 100A-100C illustrate example graphical user interface models of adevice.

FIGS. 101A-101B illustrate example screens of a graphical user interfaceof a device.

FIGS. 102A-102D illustrate example screens of a graphical user interfaceof a device.

FIGS. 103A-103D illustrate example screens of a graphical user interfaceof a device.

FIG. 104 illustrates an example menu of a graphical user interface of adevice.

FIGS. 105A-105D illustrate example menus of a graphical user interfaceof a device.

FIGS. 106A-106C illustrate example menus of a graphical user interfaceof a device.

FIGS. 107A-107C illustrate example menus of a graphical user interfaceof a device.

FIG. 108 illustrates an example menu of a graphical user interface of adevice.

FIGS. 109A-109C illustrate example menus of a graphical user interfaceof a device.

FIGS. 110A-110B illustrate examples of scrolling in a graphical userinterface of a device.

FIG. 111A-111C illustrate examples of scrolling in a graphical userinterface of a device.

FIG. 112 illustrates examples of overlay and background content in agraphical user interface of a device.

FIGS. 113A-C illustrate examples of overlay and background content in agraphical user interface of a device.

FIGS. 114A-114B illustrate example visual transition effects in agraphical user interface of a device.

FIGS. 115A-115B illustrate example visual transition effects in agraphical user interface of a device.

FIGS. 116A-116B illustrate example visual transition effects in agraphical user interface of a device.

FIGS. 117A-117B illustrate example visual transition effects in agraphical user interface of a device.

FIGS. 118A-118C illustrate example visual transition effects in agraphical user interface of a device.

FIGS. 119A-119C illustrate example visual transition effects in agraphical user interface of a device.

FIGS. 120A-120C illustrate example visual transition effects in agraphical user interface of a device.

FIGS. 121A-121B illustrate example visual transition effects in agraphical user interface of a device.

FIG. 122 illustrates an example use of a physical model in a graphicaluser interface of a device.

FIG. 123 illustrates example screens of a graphical user interface of adevice.

FIG. 124 illustrates example screens of a graphical user interface of adevice.

FIG. 125 illustrates an example method for automatic camera activationin a device.

FIG. 126 illustrates an example method for delegation by a device.

FIG. 127 illustrates example delegation models including a device.

FIG. 128 illustrates an example method for delegating by a device.

FIGS. 129A-129D illustrate example modes of a device.

FIG. 130 illustrates an example mode of a device.

FIGS. 131A-131D illustrate example modes of a device.

FIG. 132 illustrates an example method for providing augmented realityfunctions on a device.

FIG. 133 illustrates an example network environment in which a devicemay operate.

FIG. 134 illustrates an example of pairing between a device and a targetdevice.

FIG. 135 illustrates an example method for pairing a device with atarget device.

FIG. 136 illustrates example screens of a graphical user interface of adevice.

FIG. 137 illustrates an example computer system comprising a device.

DESCRIPTION OF EXAMPLE EMBODIMENTS

FIG. 1 illustrates an example embodiment of a wearable electronic device100. Device 100 includes a body 105 containing all or some of thecircuitry, structure, and display of device 100. For example, body 105may include all or some of the processing components, data storagecomponents, memory, sensors, wiring, or communication components ofdevice 100. In particular embodiments, device 100 may include a display.The display may take any suitable form or shape, such as for example acircular shape, as illustrated by circular display 110. As used herein,where appropriate, “circular display” includes substantially circulardisplays or circular-like displays, such as for example ellipticaldisplays. In particular embodiments, device 100 may include an elementabout the display. As used herein, an element about the display includesa rotatable element encircling the display or the body on or in whichthe display sits. As an example, an element may be an outer ring 115about a circular display 110. In particular embodiments, the elementabout the display may move relative to the display or body. For example,outer ring 115 may rotate relative to the body of device 100, asdescribed more fully below. In particular embodiments, device 100 mayinclude a band 120 attached to body 105. In particular embodiments,device 100 may include a sensor module, such as for example cameramodule 125 housing a camera, affixed in or to body 105 or band 125, asdescribed more fully below.

Particular embodiments of an wearable electronic device include astack-up that allows some or all of the processing and display system tofit inside the body of the device, which may be encompassed by anelement, such as an outer ring, that provides at least one way for theuser to interact with the device. In addition or the alternative,particular embodiments may include external components incorporated intothe band for additional functionality, as described more fully herein.FIG. 2 illustrates an example stack-up 200 of an wearable electronicdevice. As illustrated in FIG. 2, some or all of the components ofstack-up 200 may adopt the form of the device, which is circular in theexample of FIG. 2. Stack-up 200 may include a layer of protective glass(or other suitable transparent, solid material) 205. Other componentsmay be laminated to protective glass 205, or be attached to base 245. Inaddition or the alternative, protective layer 205 may be mechanicallyconnected to outer ring 235, or any other suitable component of the bodyof the device. Directly beneath protective glass 205 may be atouch-sensitive layer 210. Touch-sensitive layer 210 may be composed ofany suitable material and be of any suitable type, such as for exampleresistive, surface acoustic wave, capacitive (including mutualcapacitive or self-capacitive), infrared, optical, dispersive, or anyother suitable type. Touch-sensitive layer 210 may be applied directlyto protective glass 205, laminated onto it, or physically affixed to.Touch-sensitive layer 210 may be a fully two-dimensional touch surface,or may be composed of touch-sensitive regions, such as a number ofcapacitive buttons or areas. Touch-sensitive layer 210 may be connectedto processor board 215 via a flexible connector at the edge of the touchsurface, as described more fully herein

Below the touch-sensitive layer 210 may be a circular display 215, whichmay be laminated or mechanically affixed to any of the preceding orforgoing layers. In particular embodiments, lamination may reduce glareand improve display legibility by reducing internal reflections. Asdescribed more fully below, display 215 may have an outer inactive areathat may be symmetric or asymmetric. Display 215 may be positioned suchthat it is axially centered relative to protective layer 205 for avisually symmetric presentation. Display 215 may be of any suitabletype, such as for example light-emitting diode (LED), organic lightemitting diode (OLED), or liquid crystal display (LCD). In particularembodiments, display 215 may be flexible. In particular embodiments,display 215 may be partially transparent. In particular embodiments,display 215 may be translucent.

Below display 215 may be battery 220, which in particular embodimentsmay be positioned so that base 245 may be reduced in diameter withoutaffecting the size of the battery. Battery 220 may be of any suitabletype, such as for example lithium-ion based. Battery 220 may adopt thecircular shape of the device, or may adopt any other suitable shape,such as a rectangular form, as illustrated. In particular embodiments,battery 220 may “float” in the device, e.g. may have space above, below,or around the battery to accommodate thermal expansion. In particularembodiments, high-height components such as for example haptic actuatorsor other electronics may be positioned in the additional space beyondthe edge of the battery for optimal packing of components. In particularembodiments, connectors from processor board 225 may be placed in thisspace to reduce the overall height of the device.

Below battery 220 may be processor board 225. Processor board 225 mayinclude any suitable processing components, such as for example one ormore processing units, drive units, sense units, caches, memoryelements, or integrated circuits. Processor board 225 may include one ormore heat sensors or cooling units (such as e.g., fans) for monitoringand controlling the temperature of one or more processor boardcomponents. In particular embodiments, body 105 of the device may itselfact as the heat sink

Below the processor board may be an encoder 230, encircled by one ormore outer rings 235. As described more fully below, encoder 230 may beof any suitable type, and may be part of outer ring 235 or may be aseparate component, as illustrated in FIG. 2. In particular embodiments,outer ring 235 may provide the haptic feel of the detent of the outerring or position sensing of the outer ring 235. When encoder 230 is amechanical encoder separate from the device body, as illustrated in FIG.2, the encoder may support the outer ring 235. For example, inparticular embodiments encoder 230 is mounted to base 245, and theconnections to base 245 or to band 240 may pass through some portion ofthe encoder, such as, for example, the center of the encoder. Inparticular embodiments, processor board 225 and one or more layers abovemay be attached to a central post passing through encoder 235. The postmay transfer mechanical forces on components of the device to the post,which may allow components such as the processor board and the displayto be supported by the post rather than by the encoder, reducing strainon the encoder. In particular embodiments, outer ring 235 attaches tothe moveable portion of the encoder via prongs or other suitableconnections.

The device body may conclude with a base 245. Base 245 may be stationaryrelative to the one or more rotatable components of the device, such asouter ring 235. In particular embodiments, base 245 connects to band240, described more fully herein. Connections may be mechanical orelectrical, such as for example part of the circuitry linking wiredcommunication components in band 240 to processing board 225. Inparticular embodiments, connectors are positioned to avoid the encoderand the anchor points for the bands. In particular embodiments, band 240may be detachable from base 245. As described more fully herein, band240 may include one or more inner connectors 250, one or more opticalsensing modules 255, or one or more other sensors. In particularembodiments, the interior of the device, or portions of that interior,may be sealed from the external environment.

While this disclosure describes specific examples of components instack-up 200 of wearable electronic device 100 and of the shape, size,order, connections, and functionality of those components, thisdisclosure contemplates that a wearable device, such as device 100, mayinclude any suitable components of any suitable shape, size, and orderconnected or communicating in any suitable way. As merely one example,battery 220 may be placed more toward the bottom of the stack up than isillustrated in FIG. 2. As another example, the body of the device maytake any suitable form factor, such as elliptoid or disc-like asillustrated by the example of FIG. 3A, tapered on one end as illustratedby the example of FIG. 3B, or beveled or rounded at one or more edges asillustrated by the example of FIGS. 3C-3D illustrating beveled edge 315.FIG. 3E illustrates additional example form factors of the device body,such as for example bodies 320A-E having a polygonal shape with a flatprotective covering or display or a curved protective covering ordisplay. As another example, bodies 325A-D have a partially-curved shapewith a flat protective covering or display or a curved protectivecovering or display. Bodies 330A-C have a curved shape. One or moreinternal components of the device body, such as for example one or moreinternal components, may take any form factor suitable for the body inwhich they sit.

FIG. 4A illustrates an example cross section of a device body. Asillustrated, the device body has a width of D1, such as for exampleapproximately 43 millimeters. Particular embodiments may include aslight gap D4 between the outer ring and the OLED display, such as forexample a gap of up to 0.3 millimeters. Likewise, there may also be adistance between the outer ring and a glass protective covering (whichmay have a width D3, such as for example approximately 42.6millimeters), such as for example 0.2 millimeters. In particularembodiments, the gap between the glass protective covering and the outerring is greater than the gap between the display and the outer ring. Theouter ring (which may include serration) may have a width D2 of, forexample, 1.0 millimeter. FIGS. 4B-4C illustrate example set ofconnections between components of the device. FIG. 4B illustrates atouch glass 405 above a display 410. The display is attached to the topof inner body 440 with, for example, adhesive sealant 425. Displayflexible printed circuit 430 couples the display to the electronicswithin the device body. Adhesive sealing membrane 445 may be used toconnect band 450 to the device, and one or more retention rings 435 maybe used to connect outer ring 415 to the inner body 440. In particularembodiments, the retention rings may inhibit twisting of the outer ringon its vertical axis and provide physical spacing between the outer ringand the glass covering. A layer of protective glass may sit on the topof the inner body, providing an environmental seal. In particularembodiments, a retention ring may also provide an environmental seal forthe inner body. For example, FIG. 5C illustrates an example retentionring 465 attaching an outer ring to the device body and provides anenvironmental seal between the outer ring and the inner body. Inaddition or the alternative, flock-type material, possibly coated with ahydrophobe such as, for example, TEFLON, may be used to prevent waterand dirt intrusion into the cavity. As another example, the outer ringmay be sealed to the inner body with a ring of metal or plastic,preventing air (and thus water vapor or other particles) from movingthrough the cavity between the outer ring and the inner body. Gap 455allows the outer ring to move, such as for example by rotation, relativeto the inner device body. Adhesive sealant 460 attaches the display tothe body and provides an environmental seal between the display andcomponents of the inner body.

In particular embodiments, the display of the device has a circular orelliptical form, and houses a circular display unit, such as for examplean LCD display, and an OLED display. The display unit may be mountedsuch that the visible area is centrally located within the displaymodule. Should the display unit have an offset design, one or moreappropriate maskings may be used to obscure part of the display toproduce a circular and correctly placed visual outline.

In particular embodiments, a display module has an outer ring that ispart of the user interface of the device. The outer ring may rotatewhile the band holds the bottom and inside part of the device stable.FIG. 5A illustrates an example of a top-view of the device's displayrelative to other device components. Outer ring 510 may be attached tothe front surface 512 of device 508, or it may be independent of frontsurface 512. In particular embodiments, display 506 does not rotateregardless of rotation of outer ring 510 surrounding display 506. Thatmay be achieved by attaching display 506 to the portion 504 of displaymodule that is affixed to band 502, or by programming displayed contentto remain static while the display unit rotates. In the latter case,displayed content is rotated such that the visual vertical axis of theimage displayed by the display unit remains parallel to the band at alltimes.

A display module may additionally incorporate one or more sensors on ornear the same surface as the display. For example, the display modulemay include a camera or other optical sensor, microphone, or antenna.One or more sensors may be placed in an inactive area or of the display.For example, FIG. 5B illustrates device 522 with a camera module 516placed coplanar with the battery below display 520, with optical opening514 positioned under the clear section of display 520. Camera module 516may be placed between gird line connectors 518 for display 520. Anycamera or other suitable sensors may be placed coplanar with thedisplay, such as antenna 524 of FIG. 5C, which is placed is inactivearea 526. In addition or the alternative, sensors may be placed below orabove the display, may be placed in any suitable location in or on theouter body of the device, may be placed in any suitable location in orin the band of a device, or any suitable combination thereof, asdescribed more fully herein. For example, a front-facing-camera maybeplaced under the display, on the display, or above the display.

In particular embodiments, the packaging of a circular display includesan inactive area, as illustrated in FIG. 5D. In a traditional display,row drive lines powering the display are routed to the nearest lateraledge, then either routed down along the inactive areas, or connecteddirectly to the driver integrated chips along that edge. A number ofapproaches may be taken to reduce the amount of inactive area for thedisplay. For example, particular embodiments reduce the size of theinactive area by rerouting grid control lines powering the display toone edge of the display. FIG. 5D illustrates grid control lines 532routed to one edge of display 536 and connected to a connector 538routing the lines to the processing center of device 528. In thatconfiguration, inactive area 530 may be minimized.

FIG. 5E illustrates another example embodiments for reducing theinactive area of a display 554 of device 540 by creating apolygonal-type display outline, with a circular area masked in thecenter by one or more masks 550. Connectors 552 are arranged in apolygonal design. Rows 546 and columns 542 of grid lines are routed tothe nearest connector 552. In particular embodiments, connectors 552connect to a flexible circuit behind the display that carries the driverchip. Due to the reduced density of connection, the electronics of FIG.5E may be easier to connect to a flexible printed circuit board (FPCboard) and thus increases yield. In addition, by moving the driverintegrated circuit to the back of the display, one or more inactiveareas 548 can be further reduced while allowing the integrated circuitto remain on a stable and flat surface. This design is particularlysuited to OLED displays, but may be used with LCDs, given that abacklight unit (BLU) may be laminated on to the device before the FPCboard is connected. While the above example illustrates a polygonalarrangement of connectors, any suitable arrangement of connectors may beused as long as all pixels are reached by grid lines.

FIG. 5F illustrates an example physical arrangement and sizing of adisplay of a device. The device has a diameter of D4, such as forexample approximately 41.1 millimeters. The device includes one or moreinactive areas having a width D3, such as for example approximately 1.55millimeters. The device includes a visible area with a diameter D2, suchas for example approximately 38 millimeters. The device includesconnectors 568 for column lines 564 and row lines 566. Connectors 568may be coupled to the device by one or more FPC bonds 570, which have awidth of D1, such as for example approximately 0.2 millimeters.Connectors 568 may have a width D5, such as for example approximately 6millimeters. Display connector FPC 556 may be used to connected theelectronics of the display, such as for example circuitry fromconnectors 568, to driver chip 558, which may be below the display or onthe back of the device body.

FIGS. 6A-C illustrate example cross-sectional views of a device display,including manufacturing of the device. In FIG. 6A, hotbar knife 605 isused to solder the flexible printed circuit(s) 610 coupling theelectronics of the display to processing electronics of the device. Asupport 615 may be used to stabilize the FPC 610 during this process.FIG. 6B illustrates the connected FPC 620, which has been folded over(portion 625) and glued to the back of the display using adhesive 630.FIG. 6C illustrates an example finished display. FPC 645 has beenlaminated to the back of protective display glass 635, and is bent overthe front of glass 635 and is attached to the front of glass 635 viamicrobond 649. Adhesive 650 connects the FPC 645 to the device. The FPCpass over driver chip 655, which is connected to device by adhesive 650.

In particular embodiments, all processing and RF components are locatedwithin the body of the device, which may create a challenge in allowingRF signals to pass out of the device. The FPC board may additionally beattached to sides of the polygon where there is no connection to thedisplay itself to allow the mounting of strip line, stub, ceramic, orother antennae (or other suitable sensors) in the same plane as thedisplay, as illustrated in FIG. 5C. As the antenna of FIG. 5C iscoplanar with the display, interference from the dense mesh of wiring(e.g. as illustrated in FIG. 5E) from the display is reduced.

In particular embodiments, a display may be shielded fromelectromagnetic interference with the main processor board using a metalshield. In particular embodiments, the metal shield may also be used asa heat sink for the battery, and thus may improve charge or dischargerates for the battery.

In particular embodiments, an wearable electronic device may include oneor more outer elements (which may be of any suitable shape) about thedevice body. FIG. 7A illustrates an outer element by an example outerring 710 about a display 705. Outer ring may be composed of any suitablematerial, such as for example stainless steel or aluminum. In particularembodiments, outer ring 710 may be rotatable in one direction, bothdirections, or may be used in both configurations based on e.g. aswitch. In particular embodiments, one outer ring 710 may rotate in onedirection while a second outer ring 710 rotates in the oppositedirection. Outer ring 710 may be coupled to base 720 of the device by aretention ring 715. FIG. 7B illustrates outer ring 710 attached to base720 either by a Delrin ring 715A or by a sprint steel retention ring715B. Springs or clips 725 affix the rings to base 720. FIGS. 7C-Dillustrate retention ring 715 affixed to base 720 via screws 725 screwedinto corresponding posts of base 720. The device may includefasteners/spacers 730, as illustrated in FIG. 7C.

In particular embodiments, detents or encoders (which may be usedinterchangeably, where suitable) of an outer element may provide a userwith haptic feedback (e.g. a tactile click) provided by, for example, adetent that allows the user to determine when the element has been movedone “step” or “increment”, which may be used interchangeably herein.This click may be produced directly via a mechanical linkage (e.g. aspring mechanism) or may be produced electronically via a hapticactuator (e.g. a motor or piezo actuator). For example, a motor mayprovide resistance to motion of a ring, such as for example by beingshorted to provide resistance and unshorted to provide less resistance,simulating the relative high and low torque provided by a mechanicaldetent system. As another example, magnetic systems may be used toprovide the haptic feel of a detent. For example, a solenoid mechanismmay be used to disengage the detent spring or escapement as needed. Thespring or escapement provides the actual mechanical feedback. However,this arrangement allows the device to skip a number of détentes asneeded, while re-engaging the detent at exact intervals to create thesensation of detents, such as those that have changed size. As anotherexample, a rotatable outer element (such as, for example, the outerring) may be magnetized, such as by an electromagnetic used to attractthe ring at “detent” positions, increasing torque and simulating detentfeedback. As another example, a rotatable outer element may havealternating north-south poles, which repels and attracts correspondingmagnetic poles in the device body. As another example, a permanentmagnet may be used to lock the ring in place when the electromagnet isnot in use, preventing freewheeling. As another example, instead of anelectromagnet, an easily-magnetizable ferromagnetic alloy may be usedwithin a solenoid. This allows the electromagnetic field of the solenoidto “reprogram” the magnetic orientation of the core, thus maintainingthe effect of the magnetic actuation even when the solenoid itself isdisengaged. While this disclosure provides specific examples of detents,detent-like systems, and encoders, this disclosure contemplates anysuitable detents, detent-like systems, or encoders.

FIG. 8A illustrates an outer ring 805 with notches for a spring-baseddetent system etched onto the inner surface of outer ring 805. Springs820 attached to spring posts 810. Retention ring 815 may be made ofDelrin, steel, or any other suitable material, and may be segmented orsolid/continuous. FIG. 8B illustrates an example outer ring having smallnotches 830 that engage a spring-loaded element to provide hapticfeedback from the illustrated detent. In the case of an electronicfeedback system, the feedback may be produced in rapid synchrony withthe motion of the ring, and must have a sufficient attack and decay ratesuch that successive movements of the ring are distinguishable from eachother. In particular embodiments, an outer ring may be freely (e.g.continuously) rotatable, without any clicking or stepping. In particularembodiments, a ring may be capable of both continuously rotating androtating in steps/increments, based on, for example, input from a userindicating which rotational mode the outer ring should be in. The ringmay also or in the alternative rotate freely in one direction and inincrements in the other. Different functionality may occur based on therotational mode used. For example, rotating in continuous mode maychange a continuous parameter, such as e.g. volume or zooming, whilerotation in incremental mode may change a discrete parameter, such ase.g. menu items or contacts in a list, as described more fully herein.In particular embodiments, when rotating freely the ring may providehaptic feedback to the user, for example a force applied such that thering appears to rotate in viscous media (e.g. the more quickly the ringis rotate the more it resists rotation). In particular embodiments, anouter ring may be depressed or raised in the direction of the axis theouter ring rotates about, such as for example as part of a gesture or tochange rotational modes. In particular embodiments, an outer ring mayhave touch-sensitive portions.

In particular embodiments, an encoder or detent may be used to determinethe position of the outer ring relative to the device body. Particularembodiments utilize an encoder that is affixed to the device body, asillustrated by encoder 230 of FIG. 2. In particular embodiments, theencoder is part of the inner surface of the outer ring itself, asillustrated by printed optical elements 825 in FIG. 8B. In thoseembodiments, the outer ring acts as the rotating part of the encoderdirectly. An optical encoder pattern is printed onto the inner surface,and is read out by an optical module on the processing board. Theencoder on the interior of the outer ring should have sufficient opticalcontrast for detectors, and may be etched on the outer ring via e.g.printing or laser-etching. The inner and outer rings may beenvironmentally sealed with a low-friction ring (such as for example,ring 840 of FIG. 8C) made of a material such as Teflon or Delrin thatmaintains a tight fit while preventing contaminants from entering theinner part of the device. In particular embodiments, a lip on the innerring may engage a similar lip on the outer ring, allowing the two ringsto be joined while still allowing free rotation. A larger lip at thebottom of the inner ring provides further sealing by deflectingenvironmental hazards from below. As illustrated in FIG. 9, inparticular embodiments, sealing ring 915 may fit into groove 905 of thebase, which may include a grip area 910.

In particular embodiments, a retention ring connecting the outer ring tothe body of the device may have strain gages to detect pressure on theouter ring. As an example, FIG. 10 illustrates a retention ringconnected to four strain gauges (which are also connected to the innerbody) that are symmetrically placed around the ring. As used herein, thefour strain gauges may be an electronic component detecting strain. As aresult of the symmetric placing, normal motion or contact with the outerring will place mostly asymmetric strain on the outer ring, because thering merely moves relative to the device in the plane of the ring, andthus one end compresses and the opposite end elongates, as illustratedby the top ring of FIG. 10. In contrast, squeezing a larger portion ofthe outer ring will likely produce a symmetric strain on opposite pairsof strain gauges (e.g. due to elongation of the ring under pressure).The relative difference in strain between the two pairs of strain gaugesthus differentiates intentional squeezing of the outer ring from regularmotion of or contact with the outer ring. While this disclosuredescribes specific examples of the number and placement of strain gaugesin the retention ring, this disclosure contemplates placement of anysuitable number of strain gauges in any suitable component of the deviceto detect pressure on the component. As one example, strain gauges maybe placed on the band of the device or in the outer ring.

When strain is placed on a component containing strain gauges or anyother suitable strain or pressure detection system, the detected strainmay result in any suitable functionality. For example, when strain isplaced on the outer ring, such as for example by a user squeezing theouter ring, feedback may be provided to the user. That feedback may takeany suitable form, such as tactile feedback (e.g. vibration, shaking, orheating/cooling), auditory feedback such as beeping or playing aparticular user-defined tone, visual feedback (e.g. by the display ofthe device), or any other suitable feedback or combination thereof.Functionality associated with squeezing a ring is described more fullyherein, and this disclosure contemplates any suitable functionalityresulting from strain or pressure placed on and detected by any suitablecomponents.

An wearable electronic device may be attached to a band to affix thedevice to the user. Here, reference to a “band” may encompass anysuitable apparatus for affixing a device to the user, such as forexample a traditional band 1405 that can be worn around the arm, wrist,waist, or leg of the user, as illustrated by way of example in FIG. 14A;a clip 1415 for affixing to a piece of clothing, as illustrated by wayof example in FIG. 14B; a necklace or bracelet 1420 configuration, asillustrated by way of example in FIG. 14C; a keychain 1425 or otheraccessory configuration to secure the device, for example, in the user'spocket, as illustrated by way of example in FIG. 14D; or any othersuitable configuration. Each of those embodiments may include a camera1410 located on the device, on the band, or on the body. FIG. 11illustrates various embodiments for wearing the device, such as forexample around a neck as illustrated in 1105; pinned to clothing (suchas, for example, the chest as illustrated by 1110); on a belt asillustrated in 115; on an appendage (such as, for example, an arm asillustrated in 1120); on the wrist as illustrated in 1125, or in apocket as illustrated in 1130. While this disclosure describes specificexamples of bands and ways of affixing devices to a user, thisdisclosure contemplates any suitable bands or ways of affixing a deviceto a user.

In particular embodiments, sensors and corresponding electronics may beattached to a band, where appropriate. For example, the bands of FIGS.14A-14C may be suitable for housing an optical sensor. All illustrated,particular embodiments may be suited for including a touch-sensitivearea. This disclosure contemplates any suitable bands including anysuitable sensors or electronics, such as for example communicationcomponents (such as antennae), environmental sensors, or inertialsensors. In particular embodiments, the band may be detachable from thedevice, and may communicate remotely with the device when not attachedto the device. In particular embodiments, wiring associated withelectrical components in the band may also be housed in the band, forexample to minimize the volume of the device or to minimizeelectromagnetic interference with internal device components. Forexample, devices that may cause high levels of internal EMI (forexample, camera or communication systems), that may require additionalvolume (for example, battery or speaker), that may require theenvironmental seal of the main body (for example, power/data connector),or that may require additional contact with the skin of the user (forexample, biometric sensors) may benefit by housing at least some ofelectronics in a band of the device. In particular embodiments, whenwiring is contained in a band, a display module may be attached to theband such that electronic connections made to or via the band do nottwist when the outer ring is rotated. The module may use a connectorthat is user-removable, such that the display module or device body canbe removed and attached by the user at will. As an example attachment ofa band to a device, a band 1215 as illustrated in FIG. 12A may beattached to the body by being placed over one or more posts 1205 andthen affixed to those posts using fasteners (e.g. screws) 1210. Inparticular embodiments, in addition to fasteners and posts a retentionplate 1215 may be used to secured the band to device 1225, asillustrated in FIG. 12B. This disclosure contemplates any suitableinterface between the band and the device. For example, a USB interfacemay be provided between the band and the body of the device, to forexample communicate data between the device and the band or componentsof the device and components of the band. In particular embodiments, aninterface may enable a user of the device to easily detach, attach, orchange the band of the device.

This disclosure contemplates any suitable structure for connecting aband as illustrated in FIG. 14A to itself, for example when worn by auser. For example, FIG. 13A illustrates example structures for fasteningband 1305 having a camera module 1310 to a wearer of device 1300.Fasteners may include one or more snaps 1315, holes 1320 and 1335 andcorresponding components, clasps 1340, or clips 1325 with push buttons1330. FIG. 13B illustrates an example mechanism for affixing band 1301to a wearer using clips 1311 and 1303. Components 1309 insert in thecavity on the other side of components 1307 to fasten band 1301. FIG.13B further illustrates example internal mechanisms for clips 1303 and1311. Component 1317 of clip 1313 (corresponding to clip 1311) mayinclude one or more magnetic portions, which may be attracted to magnetsin cavity 1323. For example, component 1317 may include a magneticportion at its outer edge, and a magnet of opposite polarity may beplaced in front of spring 1319 to attract the magnet of component 1317.Components 1317 may then fill cavity 1323, fastening clip 1313 to clip1303 by coupling of the magnets. Once inserted, components 1321 may beused to engage springs 1319, which force components 1317 out of cavity1323. Clip 1313 may be detached from clip 1303. In addition to magnetson components 1317 and in cavity 1323, magnets may also be placed withinclip 1313, for example to assist removal of clip 1313 when springs 1319are engaged or to prevent components 1317 from sliding in and out ofclip 1313 when not fastened to clip 1303. For example, one or moremagnets may be placed in the center of clip 1313 equidistant fromcomponents 1317 and in the same plane as components 1317, attractingmagnets of each component (and thus, the components themselves) towardthe center of clip 1313.

FIG. 13C illustrates example structure for affixing a band 1327 usingfasteners 1333 and 1331, for example through the use of cavity 1329 andcomponents 1337 and 1341. FIG. 13C illustrates the internal structure offasteners 1331 and 1333. Fasteners 1339 (corresponding to fastener 1333)includes components 1337. When fastener 1343 (corresponding to fastener1331) is inserted into fasteners 1339, components 1341 attach tocomponents 1337, and may be secured by extending over a lip of fastener1339. When fastener 1339 is pulled upwards the lip increasingly forcescomponents 1337 out, moving components 1341 past the lip of fastener1339 and enabling fastener 1339 to be removed from fastener 1343. Inparticular embodiments, magnets may be placed in or on fasteners 1333and 1331 to fasten them together. For example, a magnet may be placed atthe edge of each of component 1341 and 1337. When fastener 1343 isbrought into fastener 1337 (or vice versa) the magnets attract andsecure component 1341 to component 1337. In addition, a magnet may beplaced in fastener 1343, for example to assist removal of component 1341from component 1337 or to prevent components 1341 from sliding in andout of fastener 1343 when not affixed to fastener 1339. For example, oneor more magnets may be placed in the center of fastener 1343 equidistantfrom components 1341 and in the same plane as components 1341,attracting magnets at the end of each component (and thus, thecomponents themselves) toward the center of fastener 1343.

FIG. 13D illustrates an alternative arrangement for affixing band 1351using fasteners 1349 and 1353. When affixed, fastener 1357(corresponding to fastener 1353) may be twisted, disengaging components1359 (which may be rounded) from cavities 1363, and enabling fastener1361 (corresponding to fastener 1349) to be removed from fastener 1357,and vice versa. In particular embodiments, one or magnets may be used toaffix fasteners 1357 and 1361 to each other and/or remove fasteners 1357and 1361 from each other. For example, magnets may be placed in cavities1363 and at the outer (convex) edge of components 1359, attractingcomponents 1359 into cavities 1363 and securing fastener 1361 tofastener 1357. As another example, magnets may be placed on the inneredge of components 1359 (i.e., on the concave surface of components1359), attracting components 1359 into fastener 1361, for example toassist removal of components 1359 from cavities 1363 or to preventcomponents 1359 from sliding in and out of fastener 1361 when notaffixed to fastener 1357. Corresponding magnets may also be placed onthe surfaces of fastener 1361 that are in contact with components 1359when those components are not extended into cavities 1363. In otherwords, those magnets may attract (and, in particular embodiments,ultimately make directed contact with) magnets on the concave surface ofcomponents 1359, securing components 1359 to fastener 1361.

FIGS. 13E-13G illustrate example embodiments of affixing a band 1369with camera module 1373 to itself, for example when worn by a user ofdevice 1367. In FIG. 13E, one or more magnets 1371 on one side of band1369 may be attracted to one or more magnets 1379 on the other side ofband 1369. Magnets may be strips of magnetic material partially crossinga band, as illustrated by magnetic strip 1307 in FIG. 13H, may be stripsof magnetic material fully cross the band, as illustrated by strips 1321and 1327 in FIG. 13I, or may be areas of magnetic material 1393 asillustrated in FIG. 13F. In addition to magnets 1371 and 1379, band 1369may include holes 1391 and one or more posts 1377 for securing band 1369to the wearer of device 1367. FIG. 13G illustrates fasteners 1387 (e.g.screws 1396) affixing to fasteners 1371 (e.g. nut with covering 1395) toaffix band 1381 to a wearer of device 1367 using holds 1383 (1398).

In particular embodiments, a band containing electrical components mayalso incorporate a traditional physical contact connector, asillustrated by connector 250 of FIG. 2. The connector may allow forconnectivity with the device, for example, for charging, system updates,debugging, or data transfer. Such a connector may be of the pogo varietyor may be plated surfaces to which a charging cable can interface bycontact. Such connectors may be plated in precious metals to preventcorrosion from exposure to moisture from the environment and the humanbody. In particular embodiments, physical connectors may be used onlyfor power, and data may be transferred using short-range communicationmodalities, such as BLUETOOTH, near field communication (NFC)technology, or WI-FI.

In particular embodiments, a band may be used to house flexiblebatteries (such as, e.g., lithium-based batteries) to increase theenergy storage of the device. As energy storage capacity may be tied tototal volume, batteries internal to the band increase the storagecapacity for volume-limited wearable devices without impacting the totalsize of the device body.

As described more fully below, an wearable electronic device may includeone or more sensors on or in the device. For example, an wearableelectronic device may include one or more optical sensors or depthsensors. Optical sensors may be placed in any suitable location, such asfor example on the face of the device, on a band facing outward from theuser's body, on a band facing opposite the face, on a band facing towardthe user's body, or any suitable combination thereof. FIG. 15illustrates a device 1500 with a band having an outward-facing opticalsensor 1505. Placement of an optical sensor on the band may reduce thenumber of high-frequency signals inside the case, allowing for lightershielding within the device body and thus weight and volume savings.FIGS. 14A-14D illustrate example camera placements for differentembodiments of an wearable electronic device. In particular embodiments,electronics such as that for processing camera input may be located inthe band as well, for example in a “volcano” shape housing the camera,as illustrated by housing 125 in FIG. 1. In particular embodiments,other sensors may be placed near an optical sensor, such as for examplein the same housing as the optical sensor on the band of the device. Forexample, a depth sensor may be used in conjunction with an opticalcamera to enhance display or detection of a device's environment, or todetermine which object a user is pointing at or interacting with via agesture.

In particular embodiments, placement of an optical sensor on the bandmay be adjustable by the user within a predetermined range. Inparticular embodiments, placement of an optical sensor on the band maybe optimized such that the sensor is conveniently aimable by the user.For example, as illustrated by FIG. 15 if the user wears the deviceabout the user's wrist, optical sensor 1505 may be placed in anoutward-facing fashion such that the optical sensor aims outward fromthe user's body when the user's palm is roughly parallel to the ground.

In particular embodiments, placement of an optical sensor may be suchthat the user may view the display of the device while the sensor ispointing outward from the user's body. Thus, the user may view contentcaptured by the optical sensor and displayed by the device withoutblocking the user's view of the physical scene captured by the sensor,as illustrated by the viewing triangle in FIG. 16. A display 1620 of adevice 1600 may have an associated viewing cone, e.g., the volume withinwhich the display can be reasonably viewed. In FIG. 16, user 1615 (1)views a real trophy 1610 and (2) views an image of the trophy on display1620 of device 1600 from within the viewing cone of display 1620 byaiming sensor 1605 at the real trophy. Sensor 1605 has an associatedangle of view corresponding to a volume within which images can bereasonably captured by sensor 1605. Note that in the example of FIG. 16,sensor 1605 is placed such that the user can conveniently aim sensor1605 outward while maintaining display 1620 of device 1600 in adirection facing the user, and can do so without device 1600 blockingthe user's view of trophy 1610.

FIG. 17 illustrates an example angle of view for an optical sensor. Whenobject 1725 is in the angle of view of optical sensor 1705, a user mayview both object 1725 and an image 1710 or 1715 of object 1725 asdisplayed on device 1700. For example, when the user's hand 1720 is inthe angle of view, the user may view object 1725, hand 1720, and animage 1710 of object 1725 and hand 1720 on display 1700 of the device.In contrast, when hand 1720 is not in the angle of view of sensor 1705,hand 1720 is not displayed by image 1715 presented on display 1700. Whenworn by a user, the device's sensor may capture the user'shand/arm/fingers in the angle of view of the sensor while performing agesture to be captured by the same or other sensors (e.g. a gestureselecting an object in the angle of view of the device, such as, forexample, pinching, tapping, or pulling toward or pushing away). Thesensor and display may be oriented such that, when worn by a user, anobject to be displayed on the device is in the angle of view of thedevice while the device does not block the user's view of the object andthe user's gaze is within the viewing cone of the device's display. Inparticular embodiments, a user may interact with the image captured bythe sensor or displayed on the device, such as, for example, by tappingon the portion of the display at or near where the image is displayed,by performing a gesture within the angle of view of the sensor, or byany other suitable method. This interaction may provide somefunctionality related to the object, such as, for example, identifyingthe object, determining information about the object, and displaying atleast some of the information on the display; by capturing a picture ofthe object; or by pairing with or otherwise communicating with theobject if the object has pairing/communicating capabilities.

In particular embodiments, an optical or depth sensor module (which maybe used interchangeably, where appropriate) may communicate with adevice via a simple extension of the bus the optical sensor would use ifit were directly mounted on the main printed circuit board (PCB), asillustrated in FIG. 18A. In FIG. 18A, optical sensor 1825 transmits dataover flexible printed circuits or wiring 1820 to an integrated control1810, which in the example of FIG. 18A is located in or on device 1805,which houses the main printed circuit board. FIG. 18B illustrates theoptical sensor integrated circuit 1850 on or in the optical sensormodule 1860, which also houses optical sensor 1855. Communicationbetween the main printed circuit board of device 1830 and electronics incamera module 1860 occur via flexible printed circuit 1845. Thearrangement of FIG. 18B may allow an integrated circuit to compress andotherwise process the data and send it via a method that requires fewersignal lines, or that requires a smaller transfer of data. That may bebeneficial since the band must flex when the user wears the device, andthus a smaller number of lines may be desirable. Such an approach canreduce the number of lines to one or two signal lines and two powerlines, which is advantageous for packaging, molding, and reliability. Inparticular embodiments, one or more of the electronics described abovemust be shielded to prevent electromagnetic interference from the longhigh-frequency cabling. The use of a parallel bus is common is suchcases, and may require the use of a larger cable or FPC.

In one embodiment, the camera control integrated circuit may be mounteddirectly on a small circuit board at the optical module, as illustratedin FIGS. 18A-B. An wearable electronic device may include any suitablesensors. In particular embodiments, one or more sensors or itscorresponding electronics may be located on a band of the device, in oron the body of a device, or both. Sensors may communicate with eachother and with processing and memory components through any suitablewired or wireless connections, such as for example direct electricalconnection, NFC, or BLUETOOTH. Sensors may detect the context (e.g.environment) or state of the device, the user, an application, oranother device or application running on another device. This disclosurecontemplates an wearable electronic device containing any suitableconfiguration of sensors at any suitable location of the wearableelectronic device. In addition, this disclosure contemplates anysuitable sensor receiving any suitable input described herein, orinitiating, involved in, or otherwise associated with the provision ofany suitable functionality or services described herein. For example,touch-sensitive sensors may be involved in the transition betweengraphical user interfaces displayed on the device, as described morefully herein. This disclosure further contemplates that functionalityassociated with the wearable device, activation/deactivation of sensors,sensitivity of sensors, or priority of sensor processing may beuser-customizable, when appropriate.

FIG. 19 illustrates an example sensor detection system and illustratesexample sensors for an wearable electronic device. Sensors send data ina sensor-specific format to the sensor hub subsystem of the device. Forexample, sensors 19A illustrated in example sensor module 1924 mayinclude one or more: face-detecting cameras 1902, outward-facing cameras1904, face proximity sensors 1906, face touch sensors 1908, band touchsensors 1910, acoustic skin touch sensors 1912, inertial measurementsystem (IMU) 1914, gravity vector sensors 1916, touch sensors 1918 and1920, and any other suitable sensors 1922. Data from sensors is sent tosensor hub 19B illustrated in example sensor hub module 1944. The datais conditioned and cleaned of noise in steps 1928 and 1930 as needed andtransferred to a locked-state detector 1942. Locked state detector 1942detects when the device is inactive, and disables sensors as needed toconserve power, while monitoring the sensor data for a gesture or othersuitable input that may reactivate the device. For example, numericgesture detectors receive sensor output and compare that output to oneor more numeric thresholds to determine an outcome. Heuristic gesturedetectors 1934 receive sensor output and make decisions based on one ormore decision trees, such as for example ANDing rules applied to morethan one threshold. Pattern-based gesture detectors 1938 evaluate sensorinput against a predetermined library of gesture patterns 1940, such asfor example patterns determined by empirical evaluation of sensor outputwhen a gesture is performed. One or more gesture priority decoders 1948evaluate output from gesture detectors, locked state detectors, or bothto determine which, if any, of the detected gestures should be utilizedto provide functionality to a particular application or system-levelprocess. More broadly, in particular embodiments, when the device isactive, application-requested or system-requested sensor detectors areactivated in turn and provide their data to the sensor priority decoder.In particular embodiments, the priority detector determines which, ifany, of a plurality of sensor input to process, and this disclosurecontemplates that combined input from multiple sensors may be associatedwith functionality different than functionality associated with eachsensor input individually. The decoder decides when a sensor has beendetected with sufficient certainty, and provides sensor data to thesensor hub driver. The driver provides an application programminginterface (API) to the end applications and system controllers, which inturn produce necessary output and navigation. For example, FIG. 19illustrates example sensor hub driver 1950, application APIs 1952,system navigation controllers 1954 for, for example, determiningappropriate system functionality (for example, system-level navigation1962 through a graphical user interface of the device), andapplication-level gesture priority detectors for applications 1956.While sensor hub 19B and application processor 19C (illustrated inexample application processor module 1964) of FIG. 19 are illustrated asseparate entities, they may be expressed by (and their functionsperformed by) at least some of the same or similar components. Inparticular embodiments, the boundaries delineating the components andfunctions of the sensor hub and the application processor may be more orless inclusive. The boundaries illustrated in FIG. 19 are merely oneexample embodiment. As for sensors themselves, functions executed by andcomponents of the sensor hub system and application processor may occuror be in the device body, in the band, or both. Particular embodimentsmay use more than one sensor hub or application processor, or componentstherein, to receive and process sensor data.

Sensors may internally produce sensor data, which may be simply filteredor reformatted by, for example, a detector or data conditioner. Raw datamay be formatted to an uniform format by the data formatter foringestion by the Application API. Recognizers may use numeric models(such as decision trees), heuristic models, pattern recognition, or anyother suitable hardware, software, and techniques to detect sensor data,such as gesture input. Recognizers may be enabled or disabled by theAPI. In such cases, the associated sensors may also be disabled if therecognizer is not to receive data from the sensors or is incapable ofrecognizing the sensor data.

A device may incorporate a database of sensor outputs that allow thesame detector to detect many different sensor outputs. Depending on therequests produced by the API, a sensor priority decoder may suppress orpass through sensor output based on criteria supplied. The criteria maybe a function of the design of the API. In particular embodiments,recognizers may ingest the output of more than one sensor to detectsensor output.

In particular embodiments, multiple sensors may be used to detectsimilar information. For example, both a normal and a depth sensingcamera may be used to detect a finger, or both a gyroscope and amagnetometer may be used to detect orientation. When suitable,functionality that depends on or utilizes sensor information maysubstitute sensors or choose among them based on implementation andruntime considerations such as cost, energy use, or frequency of use.

Sensors may be of any suitable type, and as described herein, may belocated in or on a device body, in or on a band, or a suitablecombination thereof. In particular embodiments, sensors may include oneor more depth or proximity sensors (terms which may be usedinterchangeably herein, when appropriate), such as for example infraredsensor, optical sensors, acoustic sensors, or any other suitable depthsensors or proximity sensors. For example, a depth sensor may be placedon or near a display of a device to detect when, e.g., the user's hand,finger, or face comes near the display. As another example, depthsensors may detect any object that a user's finger in the angle of viewof the depth sensor is pointing to, as described more fully herein.Depth sensors also or in the alternative may be located on a band of thedevice, as described more fully herein. In particular embodiments,sensors may include on or more touch-sensitive areas on the device body,band or both. Touch-sensitive areas may utilize any suitabletouch-sensitive techniques, such as for example resistive, surfaceacoustic wave, capacitive (including mutual capacitive orself-capacitive), infrared, optical, dispersive, or any other suitabletechniques. Touch-sensitive areas may detect any suitable contact, suchas swipes, taps, contact at one or more particular points or with one ormore particular areas, or multi-touch contact (such as, e.g., pinchingtwo or more fingers on a display or rotating two or more fingers on adisplay). As described more fully herein, touch-sensitive areas maycomprise at least a portion of a device's display, ring, or band. Likefor other sensors, in particular embodiments touch-sensitive areas maybe activated or deactivated for example based on context, powerconsiderations, or user settings. For example, a touch-sensitive portionof a ring may be activated when the ring is “locked” (e.g. does notrotate) and deactivated when the ring rotates freely. In particularembodiments, sensors may include one or more optical sensors, such assuitable cameras or optical depth sensors.

In particular embodiments, sensors may include one or more inertialsensors or orientation sensors, such as an accelerometer, a gyroscope, amagnetometer, a GPS chip, or a compass. In particular embodiments,output from inertial or orientation sensors may be used to activate orunlock a device, detect one or more gestures, interact with content onthe device's display screen or a paired device's display screen, accessparticular data or activate particular functions of the device or of apaired device, initiate communications between a device body and band ora device and a paired device, or any other suitable functionality. Inparticular embodiments, sensors may include one or more microphones fordetecting e.g. speech of a user, or ambient sounds to determine thecontext of the device. In addition, in particular embodiments a devicemay include one or more speakers on the device body or on the band.

In particular embodiments, sensors may include components forcommunicating with other devices, such as network devices (e.g. serversor routers), smartphones, computing devices, display devices (e.g.televisions or kiosks), audio systems, video systems, other wearableelectronic devices, or between a band and a device body. Such sensorsmay include NFC readers/beacons, BLUETOOTH technology, or antennae fortransmission or reception at any suitable frequency.

In particular embodiments, sensors may include sensors that receive ordetect haptic input from a user of the device, such as for examplepiezoelectrics, pressure sensors, force sensors, inertial sensors (asdescribed above), strain/stress sensors, or mechanical actuators. Suchsensors may be located at any suitable location on the device. Inparticular embodiments, components of the device may also provide hapticfeedback to the user. For example, one or more rings, surfaces, or bandsmay vibrate, produce light, or produce audio.

In particular embodiments, an wearable electronic device may include oneor more sensors of the ambient environment, such as a temperaturesensor, humidity sensor, or altimeter. In particular embodiments, anwearable electronic device may include one or more sensors for sensing aphysical attribute of the user of the wearable device. Such sensors maybe located in any suitable area, such as for example on a band of thedevice or on base of the device contacting the user's skin. As anexample, sensors may include acoustic sensors that detects vibrations ofa user's skin, such as when the user rubs skin (or clothing coveringskin) near the wearable device, taps the skin near the device, or movesthe device up and down the user's arm. As additional examples, a sensormay include one or more body temperature sensors, a pulse oximeter,galvanic-skin-response sensors, capacitive imaging sensors,electromyography sensors, biometric data readers (e.g. fingerprint oreye), and any other suitable sensors. Such sensors may provide feedbackto the user of the user's state, may be used to initiate predeterminedfunctionality (e.g. an alert to take particular medication, such asinsulin for a diabetic), or may communicate sensed information to aremote device (such as, for example, a terminal in a medical office).

An wearable electronic device may include one or more chargingcomponents for charging or powering the device. Charging components mayutilize any suitable charging method, such as capacitive charging,electromagnetic charging, trickle charging, charging by directelectrical contact, solar, kinetic, inductive, or intelligent charging(for example, charging based on a condition or state of a battery, andmodifying charging actions accordingly). Charging components may belocated on any suitable portion of the device, such as in or on the bodyof the device or in or on the band of a device. For example, FIG. 20Aillustrates a charger 2000 with slot 2005 for connecting a chargingcomponent with the charger. For example, slot 2005 may use friction,mechanical structures (such as latches or snaps), magnetism, or anyother suitable technique for accepting and securing a prong from acharging component such that the prong and charger 2000 make directelectrical contact. FIG. 20C illustrates prong 2015 on band 2010utilizing pogo-style connectors to create a circuit connection betweencharger 2022 and band 2010 through contacts 2020. In particularembodiments, prong 2015 may be on charger 2022 and slot 2005 of FIG. 20Amay be on the band or body of the wearable device. In particularembodiments, contacts 2020 (such as, for example, pogo-style connectors)may be on the body of the device, which may be used to create a circuitbetween the band or the charger for charging the device. Charger 2000 ofFIG. 20A may be connected to any suitable power source (such as, forexample, power from an alternating current (AC) outlet or direct current(DC) power from a USB port on a computing device) by any suitable wiredor wireless connection.

Charger 2000 may be made of any suitable material, such as acrylic, andin particular embodiments may have a non-slip material as its backing,such as e.g. rubber. In particular embodiments, charger 2000 may beaffixed or attached to a surface, for example may be attached to a wallas illustrated in FIG. 20B. Attachment may be made by any suitabletechnique, such as for example by mechanically, magnetically, oradhesively. In particular embodiments, an wearable electronic device maybe fully usable while attached to the charger. For example, when acharging component is located on the body of the device, the device maysit in the charger while a user interacts with the device or otherdevices communicate with the device.

As another example of a charging components in a wearable electronicdevice, FIGS. 21A-21B illustrate additional example chargers using e.g.inductive charger. As illustrated in FIGS. 21A-21B, a band may includeone or more charging coils 2110. As described above, this disclosurecontemplates charging coils (or any other suitable charging component)incorporated in or on the body of the device, in alternative to or inaddition to on the band of the device. A magnetic field 2105 generatedby e.g. charging surface 2115 or charging surface 2120 passes throughcharging coil 2110. Charging surface 2120 of FIG. 21B may improve thedensity of the magnetic field 2105 through charging coil 2110 relativeto charging surface 2115 and allows more precise placement than chargingsurface 2115, thus improving the charge transfer rate of the system.This disclosure contemplates that, when suitable, charging may powercomponents on or on the body of the device, components in or on theband, or both.

In particular embodiments, the band or device may implement an antennafor a wireless charging solution. Since wireless charging operatesoptimally in the absence of ferrous metals, this allows a wider choiceof materials for the body of the device, while allowing improvedwireless charging transfer capacity by allowing the coil to be heldbetween the poles of a charging driver (as described above) rather thanbeing simply coplanar to the driver. As described above and illustratedin FIG. 2, the active band may also incorporate a traditional internalphysical contact connector 250.

In particular embodiments a charging unit with an internal chargereservoir may be associated with a wearable electronic device. Whenplugged into the wall, the charging unit can charge both an attacheddevice and the charging unit's internal reservoir. When not plugged in,the charging unit can still charge an attached device from its reservoirof power until that reservoir is depleted. When only the charger isconnected to a power source without a device, it still charges itself,so that it can provide additional power for the device at a later point.Thus, the charging unit described herein is useful with and withoutbeing plugged into a power source, as it also can power anypartially-charged device for a while when a person is not able toconnect to a power source, for example when travelling, on plane, trainstation, outdoors, or anywhere a user might need charge for a device butdoes not have access to a power source. The device can be both instandby or in-use while the charger charges the device, and nomodifications to the software or hardware of the target device isneeded. Additional benefits of one or more embodiments of the inventionmay include reducing the number of items one must carry, providing thebenefits of both a charger and a power pack, making charger useful tocarry when on the move, and reducing the number of cables and connectorsone must carry to extend the battery life of their devices. Thisdisclosure contemplates that such a charging unit may be applied to anysuitable electronic devices, including but not limited to an wearableelectronic device.

FIGS. 22A-22B illustrate particular embodiments of an example chargingunit 2210 with example connections 2205 to device 2200 and connections2215 and 2220. For example, FIG. 22A illustrates cable connectivity fromthe charging unit 2210 to device 2200 and to an external power source.As another example, FIG. 22B illustrates charging unit 2210 with cableconnectivity from device 2200 and direct connectivity to a power source.This disclosure contemplates any suitable connections between a device,the charging unit, and a power source charging the charging unit. Forexample, connections both to the device and to the power source may bedirect, via cabling, or wireless.

As described above, a charging unit can charge a device from thecharging unit's internal charging reservoir even when not connected toan external power source, and can charge itself, a connected device, orboth when connected to an external power source. This disclosurecontemplates any suitable scheme for allocating charge between thecharging unit and device. Such allocation scheme may depend on theamount of charge internal to the device, internal to the charging unit,the amount of power being consumed by the device, the chargingcapabilities of an external power source, or any suitable combinationthereof. In addition or the alternative, charging threshold maydetermine which allocation scheme to use. For example, one chargingscheme may be used when the device is near full charge and the chargingunit has little charge left, and another may be used when the device haslittle charge left. FIGS. 23-24 illustrate example charging schemes forthe charging unit and connected device. For example, as illustrated inFIG. 24, when a device gets connected to a charger as in step 2400, step2405 determined whether the device is fully charged. If yes, no furthercharging action is taken. If not, step 2410 determines whether thecharger is connected to an external power source, such as for exampleline voltage. If so, the device is charged from that external source in2425. If not, step determines whether the charger has any power left,and if so, the device is charged from the charger's internal powersource in step 2420 from line voltage rather than the charging unit'sreservoir when the charging unit is connected to the line voltage. FIG.23 illustrates a similar decision tree. If a device is connected to acharger (step 2300) that is connected to a power source (step 2300),then step 2310 determines whether the device is fully charged, and ifnot, the device is charged from the power source the charger isconnected to (step 2315). Similarly, step 2320 determines whether thecharger is fully charged, and if not, the charger unit is charged fromthe power source in step 2325. In particular embodiments, the allocationscheme used may be determined or customized by a user.

FIGS. 25A-25E illustrate example embodiments of energy storage andcharging in a device and a charging unit. In FIG. 25A of the illustratedembodiment, charge reservoir 2500 of the device and charge reservoir2520 of the charging unit are both depleted. FIGS. 25B-25C illustratecharging the charge reservoir 2500 of the device and charge reservoir2505 of the device after the charging unit has been connected toexternal power source 2510. After a short time, both charging unit andthe device are charged simultaneously, with charging being distributedsuch that each is given the same percent of its total charge capacity.Both charging reservoir 2500 of the device and the charging reservoir2505 of the charging unit are completely charged after some time, asillustrated in FIG. 25C. As described herein, the amount of chargeallocated to the device or the charging unit may vary based on anysuitable charge allocation scheme. For example, if the power conversioncapability of the charging unit is limited, the charging unit'sreservoir is nearly full and the device's charge reservoir is nearlyempty, or the energy demand of the device is very high, the chargingunit may prioritize the charging of the device before charging itsinternal reserves. As another example, charging of the charging unit maycontinue until a predetermined threshold charge has been reached.

FIGS. 25D-25E illustrate transfer of charge between the charging unitand the device when the charging unit is not connected to an externalpower source. As illustrated in FIG. 25D, a device with little chargeremaining in its reservoir 2500 is connected to a charging unit with afully charged reservoir 2505. As discussed above, this disclosurecontemplates any suitable charge allocation scheme between the deviceand the charger when the charger is not connected to an external powersource. That allocation scheme may be the same as or different from theallocation schemed used when the charging unit is connected to anexternal power source. For example, FIG. 25E illustrates an allocationscheme that maximizes the charge of the charging reservoir 2500 of thedevice. As long as the charging unit still has charge, it continuescharging device until the device is fully charged or until the chargingreservoir 2505 of the charger is completely empty.

FIG. 26 illustrates an example internal architecture of an examplecharging unit 2600. Line voltage converter 2605 produces a lower voltagedirect current from the high voltage line current 2610. This voltage isfed both to battery charger/regulator 2630 and connector 2615, to whicha device can be connected via connection 2620 for charging. Batterycharger 2630 uses available power from line voltage converter 2605 tocharge the energy reservoir (battery 2635). It may take an equal shareof the power as the device, take a smaller share when the device demandis high (device priority), or take a larger share when internal powerreserves are low (charger priority). Those priorities may beuser-selectable.

Continuing the example of FIG. 26, when line voltage converter 2605 isnot providing power, charger/regulator 2630 produces the appropriatecharging voltage from the power on battery 2635. Regulator 2630 may bealways on, or it may be switched on by connection to the device, or thepress of a button that indicates the user wishes to charge the device.Once activated, regulator 2630 will charge the device until internalreserves are depleted. At that point, some charge may still remain inbattery 2635 to improve battery life, but it will not be available tothe user. The device may incorporate an emergency mode that allowsaccess to some of this energy to gain a minimal amount of emergencyusage time, at the cost of battery lifetime. Regulator 2630 may continueto provide energy until either the device is unplugged, or until thedevice only draws a minimal amount of energy, indicating completion ofcharge. Finally, charger/regulator 2630 may include an on-demand displaythat shows the amount of energy remaining in reserve to the user. Sincedisplays generally use energy, a button or other input may be used totrigger the display for a limited time. While FIG. 26 illustrates anexample internal architecture of an example charging unit 2600, thisdisclosure contemplates any suitable internal architecture of anysuitable charging unit described herein, and contemplates that such acharging unit may be of any suitable size and shape.

In particular embodiments, functionality or components of the device(such as e.g. sensors) may be activated and deactivated, for example, toconserve power or reduce or eliminate unwanted functionality. Forexample, a locked state detector detects when the device is inactivated,and disables sensors as needed to conserve power, while monitoring thesensor data for a gesture or other suitable input that may reactivatethe device. A device may have one or more power modes, such as sleepmode or fully active mode. As one example, in particular embodiments thedevice is arm-worn, and a touch surface of the device may come incontact with objects and persons while in regular use. To preventaccidental activation, an accelerometer or other inertial sensor in thebody or band of the device can be used to gauge the approximate positionof the device relative to the gravity of the Earth. If the gravityvector is detected towards the sides of the device (e.g. the device isdetermined by at the user's side or the display is determined not to bepointed at the user) the touch screen can be locked and display disabledto reduce energy use. When the gravity vector is determined to bepointing below the device (e.g. the device is roughly horizontal,resulting in a determination that the user is viewing or otherwise usingthe device), the system may power up the display and enable the touchscreen for further interactions. In particular embodiments, in additionor in the alternative to the direction of the gravity vector waking orunlocking a device, a rate of change of the direction or magnitude ofthe gravity vector may be used to wake or unlock a device. For example,if the rate of change of the gravity vector is zero for a predeterminedamount of time (in other words, the device has been held in a particularposition for the predetermined amount of time) the device may be wokenor unlocked. As another example, one or more inertial sensors in thedevice may detect a specific gesture or sequence of gestures foractivating a display or other suitable component or application. Inparticular embodiments, the encoder of the device is robust toaccidental activation, and thus can be left active so that the user maychange between selections while bringing the device up to their angle ofview. In other embodiments the encoder may be deactivated based oncontext or user input.

In addition or the alternative to power conservation, particularembodiments may lock one or more sensors, particular functionality, orparticular applications to provide security for one or more users.Appropriate sensors may detect activation or unlocking of the secureaspects of the device or of another device paired with or communicatingwith the wearable device. For example, a specific gesture performed withthe device or on a touch-sensitive area of the device may unlock one ormore secure aspects of the device. As another example, particularrotation or sequence of rotations of a rotatable ring of the device mayunlock one or more secure aspects of the device, on its own or incombination with other user input. For example, a user may turn arotatable ring to a unique sequence of symbols, such as numbers orpictures. In response to receiving the sequence of rotational inputsused to turn the rotatable ring, the display may display the specificsymbol(s) corresponding to each rotational input, as described morefully herein. In particular embodiments, the symbols used may beuser-specific (such as, e.g., user pictures stored on or accessible bythe device or symbols pre-selected by the user). In particularembodiments, different symbols may be presented to the user after apredetermined number of unlockings or after a predetermined amount oftime. The example inputs described above may also or in the alternativebe used to activate/deactivate aspects of the device, particularapplications, or access to particular data. While this disclosuredescribes specific examples of user input unlocking secure aspects of adevice, this disclosure contemplates any suitable input or combinationof inputs for unlocking any secure aspect of the device. This disclosurecontemplates that input or other suitable parameters for unlockingsecure aspects of a device or activating/deactivating components of thedevice may be user-customizable.

In particular embodiments, an wearable electronic device may detect oneor more gestures performed with or on the device. Gestures may be of anysuitable type, may be detected by any suitable sensors (e.g. inertialsensors, touch sensors, cameras, or depth sensors), and may beassociated with any suitable functionality. For example, one or moredepth sensors may be used in conjunction with one or more cameras tocapture a gesture. In particular embodiments, several depth sensors orcameras may be used to enhance the accuracy of detecting a gesture orthe background associated with a gesture. When appropriate, sensors usedto detect gestures (or processing used to initiate functionalityassociated with a gesture) may be activated or deactivated to conservepower or provide security, as described more fully above. As shownabove, FIG. 19 illustrates an example sensor detection system andprovides specific examples of gesture detection, processing, andprioritization. In particular embodiments, specific applications maysubscribe to specific gestures or to all available gestures; or a usermay select which gestures should be detectable by which applications. Inparticular embodiments, gestures may include manipulation of anotherdevice while using the wearable device. For example, a gesture mayinclude shaking another device while aiming, moving, or otherwiseutilizing the wearable device. This disclosure contemplates that, wheresuitable, any of the gestures described herein may involve manipulationof another device. While the examples and illustrations discussed belowinvolve specific aspects or attributes of gestures, this disclosurecontemplates combining any suitable aspects or attributes of the gestureand sensor described herein.

In particular embodiments, an wearable electronic device may detect oneor more gestures performed with or on the device. Gestures may be of anysuitable type, may be detected by any suitable sensors (e.g. inertialsensors, touch sensors, cameras, or depth sensors), and may beassociated with any suitable functionality. For example, one or moredepth sensors may be used in conjunction with one or more cameras tocapture a gesture. In particular embodiments, several depth sensors orcameras may be used to enhance the accuracy of detecting a gesture orthe background associated with a gesture. When appropriate, sensors usedto detect gestures (or processing used to initiate functionalityassociated with a gesture) may be activated or deactivated to conservepower or provide security, as described more fully above. FIG. 19. Asdescribed more fully above, FIG. 19 illustrates an example sensordetection system and provides specific examples of gesture detection,processing, and prioritization. In particular embodiments, specificapplications may subscribe to specific gestures or to all availablegestures; or a user may select which gestures should be detectable bywhich applications. In particular embodiments, gestures may includemanipulation of another device while using the wearable device. Forexample, a gesture may include shaking another device while aiming,moving, or otherwise utilizing the wearable device. This disclosurecontemplates that, where suitable, any of the gestures described hereinmay involve manipulation of another device. While the examples andillustrations discussed below involve specific aspects or attributes ofgestures, this disclosure contemplates combining any suitable aspects orattributes of the gesture and sensor described herein.

In particular embodiments, gestures may include gestures that involve atleast on hand of the user and an appendage on which the device is worn,such as e.g. the other wrist of the user. For example, in particularembodiments, a user may use the hand/arm on which the device is worn toappropriately aim an optical sensor of the device (e.g. a camera ordepth sensor) and may move or position the other arm/hand/fingers toperform a particular gesture. As described herein and illustrated inFIGS. 16-17, in particular embodiments the scene aimed at may bedisplayed on the device's display, such that a user can view both thereal scene, the scene as-displayed on the device, and the user'shand/arm/fingers, if in the angle of view of the. In particularembodiments, the displayed scene may include the hands/fingers/arm useddetected by the sensor and used to perform the gesture. FIGS. 27-28illustrate example gestures in which the user aims an outward-facing(e.g. away from the body of the user) sensor on the device (e.g. on theband of the device, as illustrated in the figures) and moves orpositions his other arm/hand/fingers to perform a gesture. For example,in FIG. 27, an outward sensor detects an object in the angle of view ofthe sensor 2705, an outward sensor (which may be the same sensordetecting the object) detects one or more fingers pointing at the object2710, and when the pointing finger(s) are determined to be at rest 2715,a gesture is detected 2720. Referring to FIG. 19, raw gesture datacaptured by the outward-facing camera can be conditioned and cleaned ofnoise and that data can be sent to the Heuristic Gesture Detector. TheGesture Priority Decoder processes the gesture data and determines whenthe gesture has been identified with sufficient certainty. When thegesture has been identified, the gesture is sent to the Sensor HubDriver which provides an API to the end applications and systemcontrollers.

As examples of functionality associated with this gesture, a camera mayfocus on the object, the object detected and pointed at may then appearon the display, information about that object may appear on the display,and displayed content may be transferred to another device's display(e.g. when the object is another device). FIG. 28 illustrates an examplegesture similar to the gesture of FIG. 27; however, the illustratedgesture includes the outward-facing sensor detecting a “tapping” motionof the finger(s) (e.g. that the finger(s) are moving away from thesensor). For example, the gesture of FIG. 28 may include detecting anobject in the scene of a camera (or other suitable sensor) in step 2805,detecting the finger in the scene in step 2810, detecting a lack oflateral movement of the finger in step 2815, detecting the finger tipmoving further away from the sensor in step 2820, and detecting agesture in step 2825. The gesture illustrated in FIG. 28 may provide anysuitable functionality. For example, the “tapped” object may be selectedfrom the objects displayed on the display screen.

FIGS. 29-30 illustrate example gestures where an object is detected withan outward-facing sensor along with movement of the user's fingers andhand. For example, FIG. 29 illustrates the outward-facing sensordetecting two fingers separated 2915, the two fingers coming together(e.g. in a pinching motion) 2920, and then the pinched fingers movingtowards the sensor 2925. The motion of the fingers coming together andmoving toward the sensor may occur simultaneously or in sequence, andperforming the steps in sequence (or time between steps in the sequence)or simultaneously may each be a different gesture. In FIG. 30, the twofingers illustrated are initially near together 3010, and theoutward-facing sensor detects the fingers moving apart 3020 and the handmoving away 3015. As for FIG. 30, the movement of the fingers and thehand may be simultaneous or in any suitable sequence. In addition,aspects of FIGS. 29-30 may be combined to form a gesture. For example,pinching fingers together and moving away from the sensor may be aunique gesture. In particular embodiments, the detected fingers or handmay be manipulating another device, and that manipulation may form partof the gesture. As for all example gestures described herein, thisdisclosure contemplates any suitable functionality associated withgestures illustrated in FIGS. 29-30.

FIGS. 31-32 illustrate example gestures similar to FIGS. 29-30, exceptthat here all fingers are used to perform the gesture. In FIG. 31, thefingers are detected as initially close together (e.g. in a fist) 3105,the fist is detected moving away from the sensor 3110, and the sensordetects the fist opening 3115. Again, the sequence of steps illustratedmay occur in any suitable order. FIG. 32 illustrates the reverse of FIG.31. FIGS. 31-32 may be associated with any suitable functionality. Forexample, FIG. 31 illustrates an example of sending all or a portion ofcontent displayed on the device to another device, such as thetelevision illustrated in FIG. 31. Likewise, the gesture in FIG. 32 maypull some or all of the content displayed on another device to thedisplay of the wearable device. For example, the gestures of FIGS. 31-32may be implemented when the user performs the gestures with the wearabledevice in proximity of another device, such as a smart phone, tablet,personal computing device, smart appliance (e.g. refrigerator,thermostat, or washing machine), or any other suitable device. Thedescribed functionality are merely examples of functionality that may beassociated with gestures illustrated in FIGS. 31-32, and this disclosurecontemplates that other suitable gesture may perform the describedfunctionality.

FIGS. 33-37 illustrate an outward-facing sensor detecting a hand orportion of an arm swiping in front of the sensor. In particularembodiments, swiping with the front of the hand may be a differentgesture than swiping with the back of the hand. FIGS. 33-34 illustratethe hand being swiped from right to left 3310-3315 and left to right3410-3415 across the sensor's angle of view, and FIGS. 35-37 illustratethe hand being swiped from bottom to top 3510-3515 (as well as3735-3740) and top to bottom 3610-3615 (as well as 3710-3715) across thesensor's angle of view. As illustrated, the hand may initially start inthe angle of view, pass through the angle of view, and exit the angle ofview (as illustrated in FIG. 36); may start outside of the angle ofview, pass through the angle of view, and exit the angle of view (asillustrated in FIG. 37); may start out of the angle of view, passthrough a portion of the angle of view, and remain in the angle of view(as illustrated in FIGS. 33-35); or may start in the angle of view, passthrough a portion of the angle of view, and remain in the angle of view.This disclosure contemplates the hand being swiped at other angles, suchas, e.g., entering at a 45 degree angle below and to the right of thedevice and exiting at a 45 degree angle relative to the top and to theleft of the device. Further, this disclose contemplates detecting handswipes in motions other than a straight line, such as curved swipes ortriangular swipes. This disclosure contemplates any suitablefunctionality associated with any or all of the gestures illustrated inFIGS. 33-37, such as, for example, transitioning among user interfacesdisplayed on the device or among applications active and displayed onthe device, opening or closing applications, or scrolling throughdisplayed content (e.g. documents, webpages, or images). As reiteratedelsewhere, this disclosure contemplates any suitable gesture associatedwith the functionality described in relation to FIGS. 33-37.

FIGS. 38-39 illustrate example gestures where the outward-facing sensordetects the user's hand in the angle of view 3805 and detects one ormore fingers pointing in a direction (along with, in particularembodiments, a portion of the user's hand or arm) 3815. The gesturedetected may depend on the fingers detected or direction the detectedfingers are pointed. For example, as illustrated in FIG. 38 the fingermay be a thumb pointing upwards 3820, and in FIG. 39 the finger may be athumb pointing downwards 3920. Any suitable functionality may beassociated with gestures illustrated in FIGS. 38-39, such as saving ordeleting a file locally on the device or on an associated device, orapproving or disapproving of changes made to settings or other content.

FIG. 40 illustrates an example gesture involving a shape made withmultiple fingers or a portion of the hand in the angle of view of theoutward-facing sensor. As illustrated in FIG. 40, the shape may be aring 4010, and the gesture may include fingers not involved in the shapepointing in a specific direction 4015. As illustrated in FIG. 40, agesture may include holding the shape 4020 (and possibly the otherfingers) for a predetermined amount of time.

FIGS. 41-42 illustrate example gestures including covering all or aportion of the outward-facing sensor with the user's fingers or hand.Covering the sensor from the top of the device with a thumbs-down typegesture 4105 (as illustrated in FIG. 41) may be a different gesture thancovering the sensor from the bottom of the device 4210 (as illustratedin FIG. 42) or the sides of the device. The direction of covering may bedetected by, e.g., the shape of the hand when covering the device, theorientation of the hand when covering the device, data from othersensors indicating the direction in which the outward-facing sensor isbeing covered (e.g. detecting that the display and the outward-facingsensor are covered), or any other suitable technique.

FIGS. 43-44 illustrate example gestures where one or more of the user'sfingers or portion of a hand/arm are detected in the angle of view ofthe outward-facing sensor 4305/4405, and then move within the angle ofview (or “frame”) to perform a specific gesture 4310/4320/4410/4420. Inparticular embodiments, a gesture may be any suitable movement or may bemovement in a specific pattern. In particular embodiments, a gesture maybe associated with the fingers or a portion of the hand/arm detected.For example, a single pointing finger may be associated with a gesture4305 (as illustrated in FIG. 43) or multiple fingers/a palm may beassociated with a gesture 4405 (as illustrated in FIG. 44). Inparticular embodiments, the direction of the palm (e.g. front, back, atan angle) may be detected and associated with a gesture.

FIG. 45 illustrates an example gesture include detecting a shape withmultiple fingers or the hand/arm of the user 4505, and detectingmovement of the shape in the angle of view 4510/4520. FIG. 45illustrates the shape of FIG. 40 moving throughout the outward-facingsensor's angle of view.

FIG. 46 illustrates an example gesture involving detecting one or morefingers (some or all of a user's hand/arm) and their initialorientation, and subsequently detecting the change in orientation or therate of change of orientation over time. For example, FIG. 46illustrates detecting two fingers in a angle of view at step 4605,detecting the fingers and edge of the hand in the angle of view at step4610, detecting the fingers making a “C” shape at step 4615, decoding aninitial orientation of the “C” shape at step 4620, decoding a change inorientation of the “C” shape at step 4625, determining a relativerotational value of the “C” shape at step 4630, and detecting thegesture at step 4635. This disclosure contemplates any suitable shapemade with the user's fingers/hand/arm.

FIG. 47 illustrates an example gesture that involves detecting thenumber of fingers in a particular position in the outward-facingsensor's angle of view. For example, FIG. 47 illustrates detectingfingertips in an angle of view at step 4705, such as for example oneoutstretched thumb, an outstretched thumb and a finger, or anoutstretched thumb and two fingers. The specific fingertip orientationconfiguration is detected at step 4710, and the mapping of theconfiguration to at least a numeric count of the fingers is performed instep 4715 to detect the gesture in step 4725. Each of the displayedimages may be a different gesture. This disclosure contemplates anysuitable position of the fingers that comprise a gesture. As for allother example gesture described herein, this disclosure contemplates anysuitable functionality associated with the gestures. For example, eachgesture of FIG. 47 may be associated with a contact to call, e-mail, ortext and the detected gesture may activate the call, e-mail, or text tothe contact assigned to the gesture. In particular embodiments, theposition of the hand/arm/fingers may indicate which method of contactshould be used for the contact associated with the gesture.

FIGS. 48-49 illustrate example gestures involving dual sensors on thedevice. For example, FIG. 48 illustrates a sensor on the bottom bandportion of the device. That sensors detects the position of the user'sother hand relative to the device, and detects separation of the handfrom the sensor. In particular embodiments, the gesture may includedetermining that both hands are moving, such as for example byadditional information supplied by one or more inertial sensors in thedevice or by an inward-facing (e.g. facing the body of the user) cameradetecting movement of the device via change in scenery. For example, inFIG. 48 a hand is detected in the angle of view at step 4805. A sensordetects that the hand is in a pinched shape at step 4810 and the same oranother sensor detects that the device is in a horizontal orientation instep 4815. A sensor detects the hand moving relative to the device atstep 4820 and estimates the relative position at step 4825. The gestureis detected at step 4830. Similarly, FIG. 49 illustrates an examplegesture also involving detection of the user's hand in the angle of viewand subsequently moving away from a device sensor. However, in FIG. 49the device sensor is positioned on the top of the device (e.g. afront-facing sensor). As an example, a hand is detected in the angle ofview of a front-facing camera in step 4905. The hand is detected in apinched shape in step 4910, and the device is detected in a horizontalorientation in step 4915. The hand moves closer or further from thedevice in step 4920, and the relative position estimate is performed instep 4925, at which point the gesture is detected in step 4930.

FIGS. 50-58 illustrate example gestures detected by at least onefront-facing sensor (e.g. sensor on the top of the device.). Any of thegestures of FIGS. 50-58 may be detected by sensors in any other suitablelocation (e.g. outward-facing, as described above), and any of thegestures detected by a sensor described in another location may bedetected by a front-facing sensor, where appropriate. FIG. 50illustrates an example gesture involving one or more fingertips hoveringabove the device, and the front-facing sensor detects the fingertips instep 5005, detects the position of the fingertips or motion (or lack ofmotion) of those fingertips in steps 5010 and 5015 to detect a gesturein step 5020. FIG. 51 illustrates an example gesture in which steps 5105and 5110 are identical to 5005 and 5010, respectively. However, thedetected fingertips move away from the front-facing sensor in step 5115;in particular embodiments, a gesture may include detecting one or moreof the fingertips changing position relative to each other, such as forexample moving apart as in step 5120. FIG. 52 illustrates the fingertipsdetected by the sensor in step 5205, the fingertips moving together instep 5210, the fingers moving toward the device in step 5215, and theduration of which the motion lasts in step 5220 to detect the gesture instep 5225. As illustrated in FIG. 53, in particular embodiments agesture may include detecting a change in relative position of thefingertips in addition to the motion of the fingertips toward thesensor. For example, in step 5305 one or two fingers are detect on thefront surface; in step 5310 the fingers are detected moving upward ordownward; and a gesture is detected in step 5315. In particularembodiments, the duration of the gesture of FIGS. 50-52 may determinewhether a gesture is detected, or different durations may comprisedifferent gestures.

FIGS. 54-57 illustrate example gestures involving motion of one or morefingers or motion of a portion of a hand/arm across the face of thedevice (and thus across the front-facing sensor). As illustrated, agesture may depend on the number of fingers used (e.g. two fingers vs. awhole palm); on the direction of motion across the device face (e.g.bottom to top or left to right); on the duration of motion across thedevice face; on the proximity of the detected fingers or hand/arm to thedevice face; on the portion of the device face (e.g. all or a portion,and the relative location of the portion (e.g. bottom half)); or whetherthe detected portions are initially in the front-facing sensor's angleof view, initially out of the angle of view, end in the angle of view,or end out of the angle of view. For example, the gesture of FIG. 54 mayinclude detecting one or two fingers detected on the front surface instep 5405; detecting the fingers moving left in step 5410, and detectingthe gesture in step 5415. As another example, FIG. 55 may includedetecting one or two fingers detected on the front surface in step 5505;detecting the fingers moving right in step 5510, and detecting thegesture in step 5515. As another example, FIG. 56 may include detectingno fingers in step 5605, detecting multiple fingers entering the angleof view from the left, detecting the front surface covered, detectingthe fingers exiting the frame in step 5620, and detecting a gesture instep 5625. As yet another example, FIG. 57 may include detecting nofingers in step 5705, detecting multiple fingers entering the angle ofview from the right in step 5710, detecting a covering of the full frontsurface in step 5715, detecting the fingers exiting the angle of view instep 5720, and detecting a gesture in step 5725. As with all gesturesdescribed herein, any suitable combination of those factors (and anyother suitable factors associated with the gestures) may be used todetermine a gesture or functionality corresponding to the gesture. Anysuitable functionality may be associated with a gesture, such as, forexample, transitioning between graphical user interface screens,scrolling through displayed content, or scrolling through availableapplications or devices to communicate/pair with.

FIG. 58 illustrates an example gesture involving one or more fingersdetected on the edge of the device, and may include movement of thosefingers around all or a portion of the edge of the device. For example,as illustrated in FIG. 58, a gesture may include detecting no fingers instep 5805, detecting a single finger at the edge of the front face instep 5810, detecting a finger moving along the edge in step 5815,decoding the angular motion of the finger relative to the device in step5820, and detecting a gesture in step 5825. As an example offunctionality associated with this gesture, the movement of the fingermay rotate some or all of the displayed content on the device.

In particular embodiments, a gesture may include a motion of thewearable device, such as, for example, by the arm wearing the device.The motion may be detected by any suitable sensors, such as inertialsensors, orientation sensors, or any suitable combination thereof. FIGS.59-66 illustrate example gestures involving detection of the gravityvector relative to the device (e.g. pointing in the direction of thedevice face or pointing down through the base) and detecting subsequentmotion of the device relative to that gravity vector. For example, FIG.59 may include detecting the gravity pointing downward through the facein step 5905, detecting acceleration of the device along the same axisas the gravity vector is pointing in step 5910, detecting that theacceleration of the device remains for some time step in step 5915, anddetecting a gesture in step 5920. FIG. 60 is substantially similar tothe gesture of FIG. 59, except that the gravity vector points downthrough the base (rather than the face) in step 6005. FIG. 61illustrates a gesture that uses a gravity vector to determineorientation/position of the device, for example, that the device is notby the user's body. Motion of the device from the detected orientation(such, as for example, perpendicular to the gravity vector) may bedetected, resulting in a gesture. For example, a detected gravityorientation may indicate that an arm is not by the side of the body instep 6105, a lateral acceleration of the device may be detected in step6110, the acceleration may be detected for some time in step 6115, and agesture may be detected in step 6120. As FIGS. 59-61 indicate, detectingan aspect of the motion (e.g. duration of acceleration) may trigger agesture, and ranges of an aspect (ranges of duration of motion) may eachcorrespond to a different gesture. FIGS. 62-63 illustrate rotationalmotion of a device. As in FIG. 61, detection of the initial orientationor position of the device may be part of the gesture detection. Forexample, the gesture of FIG. 62 may include detecting that the gravityvector indicates the arm is not by the side of the body in step 6205,detecting some rotational motion in step 6210, estimating that theradius of the rotational motion is large enough for elbow motion in step6215, estimating the relative rotation in step 6220, and detecting agesture in step 6225. As another example, the gesture of FIG. 63 mayinclude detecting that the gravity vector indicates the arm is not bythe side of the body in step 6305, detecting some rotational motion instep 6310, estimating that the radius of the rotational motion is smallenough for wrist motion in step 6315, estimating the relative rotationin step 6320, and detecting a gesture in step 6325. As illustrated inFIGS. 62-63, a gesture may include estimating the type of rotation ofthe device, such as, for example, rotation primarily from the shoulder(FIG. 62), rotation primarily from the elbow (FIG. 63), or any othersuitable rotation. In addition or in the alternative to the radius ofrotation, a gesture may include detecting the amount of rotation,duration of rotation, radial acceleration of the rotation, any othersuitable aspect of the rotation, or any suitable combination thereof.

Like for FIGS. 61-63, FIG. 64 indicates a gesture involving detectingthe initial orientation or position of the device. For example, thegesture of FIG. 64 may include detecting the gravity vector indicatesthat the arm is not by the side of the body in step 6405, detectinglateral acceleration of the arm along the axis of the arm in step 6410,detecting that the acceleration remains for some time in step 6415, anddetecting a gesture in step 6420. FIG. 65 illustrates that a gesture mayinclude motion of the device along the axis of the appendage wearing thedevice, such as, for example, the acceleration of the device along thataxis. The gesture may include an impact along the path of motion (e.g.caused by the hand stopping or contacting an object) and subsequentreversal of the motion. The back-and-forth motion may repeat until themotion stops or the hand returns to some position, such as, e.g., theuser's side. In particular embodiments, different gestures may be basedon the number or frequency of the back-and-forth motion. For example,the gesture of FIG. 65 may include detecting the gravity vectorindicates that the arm is not by the side of the body in step 6505,detecting that the hand is in motion in step 6510, detecting an impulse(impact) along the path of the motion in step 6515, detecting that thehand reversed motion along the same linear path in step 6520, repeatingsteps 6515 and 6520 as suitable, detecting that the motion stops forsome time in step 6525, and detecting a gesture in step 6530.

FIGS. 66-68 illustrate example gestures based on detection of motionthat matches a predetermined motion template, which may beuser-customizable or user-creatable. In particular embodiments,customizable gestures may include an initial position or orientation ofthe device, motion or aspects of motion in a particular direction,stopping and starting of motion, duration of motion, or any othersuitable motion parameter. Some or all of the parameters may beuser-customizable, in particular embodiments. In particular embodiments,a detected gesture may be determined by matching the detected motion tothe closest available motion template. For example, as illustrated inFIGS. 66-68, a gesture may correspond to a horizontal position or motionof the arm or fingers. For example, as illustrated in FIG. 66, a gesturemay include detecting a gravity vector oriented down through the bottomof the base of the device in step 6605, detecting motion forward andinward in step 6610, matching a motion template in step 6615 (forexample, using heuristic, numeric, or pattern-based gesture recognitionmodules of FIG. 19), and detecting a gesture in step 6620. FIG. 67 mayinclude detecting a gravity vector oriented sideways through the bottomof the base of the device in step 6705, detecting motion forward andinward in step 6710, matching a motion template in step 6715 (forexample, using heuristic, numeric, or pattern-based gesture recognitionmodules of FIG. 19), and detecting a gesture in step 6720. FIG. 68 mayinclude detecting a gravity vector indicating an arm is not by the sideof the body in step 6805, detecting motion of the device in step 6810,detecting motion stopping in step 6815, matching a motion template instep 6820, selecting the best motion-template match in step 6825, anddetecting a gesture in step 6830. While FIGS. 66-68 illustrate specificexamples of customizable gestures corresponding to specific motiontemplates, this disclosure contemplates any suitable gestures (or anyaspect thereof) detected by any suitable sensors being customizable by auser of the device.

In particular embodiments, gesture may optionally include detecting somenon-motion or non-orientation input. For example FIGS. 69-71 illustratea gesture comprising detection of acoustics, although the gesturesillustrated do not require such detection. FIG. 69 illustrates anacoustic output (such as, e.g., ringing from an incoming or outgoingtelephone call) or response, followed by some motion of the device (suchas the device being brought to a user's face). For example, an audioresponse or output is initiated in step 6905, upward motion is detectedin step 6910, stopping of upward motion is detected in step 6915, thegravity vector is within a predetermined window in step 6920, and agesture is detected in step 6925. In particular embodiments, a gesturemay include detecting the gravity vector in a particular orientation ororientation window, as illustrated. The gesture of FIG. 69 may alsoinclude detecting the position of the user's hand/fingers. As an exampleof functionality that may be associated with the gesture illustrated inFIG. 69, if the fingers are brought near the ear or face in the positionindicated, the user may answer or place a telephone call. FIG. 70 andsteps 7005-7025 illustrates an example gesture having similar attributesas those described for FIG. 69, but involving different orientation ofthe user's hand/fingers. FIG. 71 illustrates an example gestureincluding acoustics generated by the user (e.g. by the user snapping herfingers together), which are detected by a microphone associated withthe device. For example, FIG. 71 may include detecting a gravity vectorindicating an arm is not by the side of the body in step 7105, detectinga motion with relatively high acceleration in step 7110, detecting asudden change in one or more acoustic frequencies in step 7115, anddetecting a gesture in step 7120. As illustrated in FIG. 71, the snapmotion may be detected solely by the motion generated by the snap alone(e.g. by the vibration of the user's hand/skin or by some degree or rateof change of rotation due to the snap), or may be detected by thecombination of motion plus an auditory input generated by the snap. Inparticular embodiments, the auditory confirmation must be detectedwithin a predetermined time of the motion for the gesture to bedetected.

FIGS. 72-73 illustrate example gestures involving periodic motion of thedevice, such as shaking of the arm the device is on in the lateral orvertical direction. FIG. 72 illustrates a gesture including detectingthe gravity vector indicating the arm is not beside the body in step7205, detecting the device moving laterally forward on an axis in step7210, detecting the device moving backwards on the same axis in step7215, repeating the steps of 7210 and 7215 as is desirable, anddetecting a gesture in step 7220. FIG. 73 illustrates a gestureincluding detecting the gravity vector indicating the arm is not besidethe body in step 7305, detecting the device moving vertically forward onan axis in step 7310, detecting the device moving backwards on the sameaxis in step 7315, repeating the steps of 7310 and 7315 as is desirable,and detecting a gesture in step 7220. FIG. 74 illustrates an examplegesture involving an adjustment of the position/orientation of thedevice relative to the user's body. For example, the gesture of FIG. 74may include including detecting the gravity vector indicating the arm isbeside the body in step 7405, detecting the gravity vector indicatingthe arm is beside the body in step 7410, detecting a gesture in step7415. Any suitable functionality may be associated with the gestures ofFIGS. 72-75, such as, for example, waking the device from a low-powerstate. FIG. 75 illustrates an example gesture involving the height ofthe device or the relative change in height of the device from start tostop of the device. In addition to the height of the device, a gesturemay include the orientation of the device before, during, or after thegesture. For example, a gesture may include detecting the gravity vectorindicating the arm is not beside the body in step 7505, detecting upwardmotion in step 7510, detecting halt of upward motion in step 7515,detecting that the gravity vector points through the side of thedevice's base in step 7520, and detecting a gesture in step 7525. Anysuitable functionality may be associated with the gesture of FIG. 75,such as, for example, activating equipment paired with the device,turning on one or more lights in a room, or activating equipment nearthe device.

In particular embodiments, a gesture may include interacting directlywith the body or band of a wearable device. For example FIG. 76illustrates a gesture involving contact with a touch-sensitive area of aband worn about the user's wrist. The gesture may include detecting thatthe device is not in a locked state in step 7605, detecting an absenceof touch on a band in step 7610, detecting touch on the band in step7615, decoding the position of the ouch in step 7620, and detecting agesture in step 7625. FIG. 77 illustrates that touches in multiplepositions may be determined to be a single gesture, such as, forexample, to unlock a device or aspects of the device. The gesture mayinclude detecting that the device is not in a locked state in step 7705,detecting an absence of touch on a band in step 7710, detecting touch onthe band in step 7715, decoding the position of the ouch in step 7720,decoding an action in step 7725, and detecting a gesture in step 7730.FIG. 78 illustrates that a gesture may include contacting atouch-sensitive area of a device and sliding across a touch-sensitivearea while maintaining contact with the device. The gesture may includedetecting that the device is not in a locked state in step 7805,detecting an absence of touch on a band in step 7810, detecting touch onthe band in step 7815, detecting movement of the touch point(s) in step7820, decoding relative motion in step 7825, and detecting a gesture instep 7830. In particular embodiments, a gesture may include the durationof contact, physical area of contact (e.g. with one finger or twofingers), the sequence of contact, pressure generated by contact, or anyother suitable contact-related attribute. While FIGS. 76-78 illustratecontact with a touch-sensitive area on a band, this disclosurecontemplates that a gesture may involve contact on a touch-sensitivearea on any suitable location of the device, such as the device band,ring, display, or any suitable combination thereof. For example, FIGS.79-80 illustrate contact with touch sensitive areas on a ring of thedevice, similar to the gestures of FIGS. 77-78. For example, a gesturemay include detecting that the device is not in a locked state in step7905, detecting lack of touch on a ring in step 7915, detecting touch onthe ring in step 7920, and detecting a gesture in step 7925. As anotherexample, a gesture may include detecting that the device is not in alocked state in step 8005, detecting lack of touch on a ring in step8010, detecting touch on the ring in step 8015, detecting movement ofthe touch point in step 8020, decoding relative motion in step 8025, anddetecting a gesture in step 8030. FIG. 81 illustrates a gestureinvolving multi-touch contact with a touch-sensitive area of a deviceface, and detecting subsequent motion of the contact points, caused by,e.g., motion of the fingers contacting the touch-sensitive area or bymovement of the wrist/hand on which the device is worn. The gesture mayinclude detecting that the device is not in a locked state in step 8105,detecting lack of touch on a surface in step 8110, detecting at leasttwo fingers touching the surface in step 8115, detecting movement of thetouch points in step 8120, decoding relative motion in step 8125, anddetecting a gesture in step 8130. Motion of the wrist/hand may bedetected by, e.g., inertial sensors in the device, allowing thedifferent ways of moving touch points to be two distinct gestures. FIG.82 illustrates a gesture involving initial contact with a device, whichmay detected by one or more proximity sensors on or in the device, orinertial sensors on or near the device. The gesture may involvedetecting that the contact persists, indicating that, e.g., the user hasput the device on. For example, the gesture may include detecting nocontact with the rear or band proximity sensor in step 8205, detectingcontact by the proximity sensor in step 8210, detecting that the contactpersists in step 8215, and detecting a gesture in step 8220. The gestureof FIG. 82 may unlock or power on a sleeping device, or provide anyother suitable functionality.

In particular embodiments, a gesture may include contact with skin nearthe device. FIG. 83 illustrates a gesture involving tapping on the skinnear where the device is worn. The tapping may be detected by vibrationsensors in the device. The tapping motion may be confirmed by, e.g., oneor more acoustic sensors detecting sound generated by the tappinggesture. For example, the gesture may include detecting that the deviceis unlocked in step 8305, detecting motion with a relatively highacceleration in step 8310, detecting the sound of, for example, a tap instep 8315, matching the motion or sound to a pattern in step 8320, anddetecting a gesture in step 8325. FIG. 84 illustrates a gestureinvolving swiping of the skin near the device, which may be detected andconfirmed by the sensors described in FIG. 83, above. For example, thegesture may include detecting that the device is unlocked in step 8405,detecting motion with a relatively high acceleration in step 8410,detecting the sound of, for example, a tap in step 8415, detecting thevibrations or sound of lateral movement on the skin in step 8420,matching the motion or sound to a pattern in step 8425, and detecting agesture in step 8430.

In particular embodiments, gestures may involve detecting metaphoricgestures made by the hand not wearing the device. For example, suchgesture may be detected by, e.g., any suitable front-facing sensor on ornear the display of the device oriented such that the hand not wearingthe device is in the angle of view of the sensor. FIG. 85 illustrates anexample gesture involving a front-facing sensor detecting motion ofmultiple fingers, such as tapping of the fingers. For example, thegesture may include determining that the device is in a predeterminedorientation in step 8505, detecting a fingertip in step 8510, detectingmotion of the fingertip in step 8515 or detecting a tap sound in step8525, and detecting one or more gestures in steps 8520 and 8530. FIG. 86illustrates an example gesture involving motion of a single finger. Forexample, the gesture may include determining that the device is in apredetermined orientation in step 8605, detecting a fingertip in step8610, detecting motion of the fingertip in step 8615 or detecting a tapsound in step 8525, and detecting one or more gestures in step 8620.FIG. 87 illustrates a gesture involving detecting movement of a handholding an object, detecting the motion of the object, locking on to theobject, and then detecting subsequent motion of the object. As aspecific example, the gesture may include detecting that the device isin a predetermined orientation in step 8705, detecting a hand in step8710, detecting motion of the hand in step 8715, detecting an additionalobject to be moving the hand in step 8720, locking on the object in step8725, detecting motion of the object in step 8730, and detecting agesture in step 8735. For example, an object may be a pen or otherstylus-like implement, and the front-facing sensor on the device maydetect writing motions of the implement to, e.g., generate/store text onthe device or on another device communicating with the wearable device.The example of FIG. 87 may allow a user to generate drawings, notes, orother written content without actually generating written content on adisplay or other writing surface. As described more fully herein, anysuitable gesture or combination of gestures may be used to impact orinitiate augmented-reality (“AR”) functionality, and may be used toperform tasks using AR functionality. For example, the gestures of FIGS.85-87 may used to capture a user's interaction with a virtual keyboard,virtual mouse, or virtual touchscreen and those interactions maygenerate input on the wearable device or any other suitable paireddevice. While this disclosure describes specific examples of metaphoricgestures and object detection (and associated functionality), thisdisclosure contemplates any suitable metaphoric gestures, detection ofany suitable objects, and such gestures associated with any suitablefunctionality.

In particular embodiments, a gesture may involve the entire appendage onwhich a device is affixed or worn. For example, FIGS. 88-92 illustrateexample gestures involving motion of the arm on which the device isworn. The gestures may include detecting the initial position of the arm(e.g. via an accelerometer detecting the direction of the gravityvector), detecting the motion of the device (via the arm), detecting thecorresponding change in the gravity vector, and detecting that the armhas stopped moving. Such gestures may also include detecting theduration of movement, the amount of movement (e.g. detecting a largeradius of motion, confirming that the entire arm has moved), theacceleration of movement, or any other suitable movement-relatedattributes. As illustrated by FIGS. 88-92, gestures may involvedetecting arm movements above the head, to the front, to the side, tothe back, or down from an initially-higher starting position. Forexample, a gesture may include detecting a gravity vector indicating ahand is on the side of the body in step 8805, detecting upward movementof the hand in step 8810, detecting that the gravity vector indicatesthe hand is above the head in step 8815, detecting the hand stoppingmovement in step 8820, and detecting a gesture in step 8825. As anotherexample, a gesture may include detecting a gravity vector indicating ahand is on the side of the body in step 8905, detecting upward andforward movement of the hand in step 8910, detecting that the gravityvector indicates the hand is horizontal in step 8915, detecting the handstopping movement in step 8920, and detecting a gesture in step 8925. Asanother example, a gesture may include detecting a gravity vectorindicating a hand is horizontal in step 9005, detecting the hand movingdownward and backward in step 9010, detecting that the gravity vectorindicates the hand is by the side in step 9015, detecting the handstopping movement in step 9020, and detecting a gesture in step 9025. Asanother example, a gesture may include detecting a gravity vectorindicating a hand is by the side of the body in step 9105, detecting thehand moving upward and backward in step 9110, detecting that the gravityvector indicates the hand is horizontal in step 9115, detecting the handstopping movement in step 9120, and detecting a gesture in step 9125. Asanother example, a gesture may include detecting a gravity vectorindicating a hand is by the side of the body in step 9205, detecting thehand moving upward and outward in step 9210, detecting that the gravityvector indicates the hand is horizontal in step 9215, detecting the handstopping movement in step 9220, and detecting a gesture in step 9225. Inparticular embodiments, gestures may involve motion of the entire bodyrather than just of the appendage on which the device is worn.

In particular embodiments, a user may interact with the device via avariety of input mechanisms or types including, for example, the outerring, touch-sensitive interfaces (e.g. the touch-sensitive layer),gestures performed by the user (described herein), or a speech interface(e.g. including voice input and speech recognition for applicationsincluding text input, communication, or searching). Additionally, inparticular embodiments, a user may interact with a graphical userinterface presented on a circular display of the device via any of theinput mechanisms or types.

A user of the wearable electronic device may interact with the device(including, e.g., a graphical user interface presented on the circulardisplay) by using the outer ring. In particular embodiments, the outerring may be touch-sensitive, such that a user's touch on one or moreportions of the ring may be detected as an input to the device andinterpreted, causing one or more actions to be taken by the device (e.g.within a graphical user interface of the device). As an example, atouch-sensitive outer ring may be a capacitive ring or inductive ring,and a user of the device may perform any suitable touch gesture on thetouch-sensitive ring to provide input to the device. The input may, forexample, include swiping the ring with one finger, swiping the ring withtwo or more fingers, performing a rotational gesture with one or morefingers, or squeezing the ring. In particular embodiments, the outerring may be rotatable, such that a physical rotation of the ring mayserve as an input to the device. Additionally, in particularembodiments, the outer ring may be clicked (e.g. pressed down) orsqueezed. Any of the embodiments of the outer ring may be combined, assuitable, such that the ring may be one or more of touch-sensitive,rotatable, clickable (or pressable), or squeezable. Inputs from thedifferent modalities of the outer ring (e.g. touch, rotation, clickingor pressing, or squeezing) may be interpreted differently depending, forexample, on the combination of the modalities of input provided by auser. As an example, a rotation of the outer ring may indicate adifferent input than a rotation in combination with a clicking orpressing of the ring. Additionally, feedback may be provided to the userwhen the user provides input via the outer ring, including hapticfeedback, audio feedback, or visual feedback, described herein.

FIG. 93A illustrates an example of a user clicking (e.g. pressing down)on the outer ring, indicated by arrows 9310. FIG. 93B illustrates anexample of a user squeezing the outer ring, indicated by arrows 9320.FIG. 94A illustrates an example of a user rotating the outer ring, suchthat content 9410 of a graphical user interface of the device changes inaccordance with the rotation (e.g. to the right). FIG. 94B illustratesan example of a user performing a rotating gesture on a touch-sensitivering, without the ring itself rotating, such that content 9420 of agraphical user interface of the device changes in accordance with therotation (e.g. to the right). FIG. 94C illustrates an example of a userrotating the outer ring while simultaneously pressing or clicking thering, such that content 9430 of a graphical user interface of the devicechanges in accordance with the rotation (e.g. to the right) and thepressing or clicking.

In particular embodiments, a touch-sensitive interface of the device(e.g. the touch-sensitive layer) may accept user touch input and allowthe device to determine the x-y coordinates of a user's touch, identifymultiple points of touch contact (e.g. at different areas of thetouch-sensitive layer), and distinguish between different temporallengths of touch interaction (e.g. differentiate gestures includingswiping, single tapping, or double tapping). Touch gestures (describedherein) may include multi-directional swiping or dragging, pinching,double-tapping, pressing or pushing on the display (which may cause aphysical movement of the display in an upward or downward direction),long pressing, multi-touch (e.g. the use of multiple fingers orimplements for touch or gesturing anywhere on the touch-sensitiveinterface), or rotational touch gestures. FIG. 95A illustrates anexample of a user tapping 9510 a touch-sensitive interface (e.g. thetouch-sensitive layer) to provide input to the device. The precise x-ycoordinates of the user's tapping may be determined by the devicethrough input from the touch-sensitive interface (e.g. thetouch-sensitive layer). FIG. 95B illustrates an example of a userperforming, respectively, a clockwise rotational gesture 9515, acounter-clockwise rotational gesture 9520, a vertical swipe gesture9525, and a horizontal swipe gesture 9530. FIG. 95C illustrates anexample of a user touching the display (including a touch-sensitivelayer with multi-touch sensing capability) using, respectively, one,two, or three points of contact 9535 (e.g. with one, two, or threefingers or implements) simultaneously. FIG. 95D illustrates an exampleof a user performing touch gestures having multiple points of contactwith the touch-sensitive interface. The user may, in this example,perform an expanding gesture 9540, a pinching gesture 9545, a clockwiserotational gesture 9550, or a counter-clockwise rotational gesture 9555with two fingers.

In particular embodiments, a graphical user interface of the device mayoperate according to an interaction and transition model. The model may,for example, determine how modes including applications, functions,sub-modes, confirmations, content, controls, active icons, actions, orother features or elements may be organized (e.g. in a hierarchy) withina graphical user interface of the device.

In one embodiment, the graphical user interface (GUI) includes multipletop-level screens that each correspond to a different mode orapplication (or sub-mode, function, confirmation, content, or any otherfeature) of the device. Each of these applications may be on the samelevel of the hierarchy of the interaction and transition model of theGUI. FIG. 96A illustrates an example layout of a hierarchy within theGUI in which multiple top-level screens 9602-9606 and 9610-9614 eachcorrespond to a different application, and one of the top-level screens9608 (the home screen) corresponds to a clock. State transitions withinthe GUI may be events triggered by input from an input source such asthe user of the device. An input from a user of the device or fromanother input source (e.g. via any of the variety of input mechanisms ortypes including the outer ring, touch-sensitive interfaces, gestures,speech, or sensors) may cause a transition within the GUI (e.g. from onetop-level screen to another). For example, an input may cause the GUI totransition from the home screen 9608 (e.g. the clock) to an application(e.g. 3 or 4) or from an application to another application. If the userrotates the outer ring to the right, for example, the GUI may transitionfrom the home screen 9608 to Application 4 9610, and if the user rotatesthe outer ring to the left, the GUI may transition from the home screen9608 to Application 3 9606. In yet other embodiments, context (e.g. asdetermined by sensors or other input sources on the device) may causethe GUI to transition from the home screen to an application or from anapplication to another application.

In one embodiment, the model may include operability for differentiationof the “left” and “right” sides in relation to the home screen. As anexample, one or more of the top-level screens may be associated withmodes or applications (or other features) in the hierarchy of theinteraction and transition model of the GUI that are fixed (e.g. alwaysavailable to the user) or contextual or dynamic (e.g. availabledepending on context). The contextual screens may, for example, reflectthe modes, applications, or functions most recently used by the user,the modes, applications, or functions most recently added (e.g.downloaded) by the user, ad-hoc registered devices (that may, forexample, enter or exit the communication range of the device as it isused), modes, applications, or functions that are “favorites” of theuser (e.g. explicitly designated by the user), or modes, applications,or functions that are suggested for the user (e.g. based on the user'sprior activity or current context). FIG. 96B illustrates an examplelayout of a hierarchy within the GUI in which contextual or dynamicapplications 9616-9620 and fixed applications 9624-9628 are groupedseparately, with the left side (in relation to the home clock screen9622) including contextual applications, and the right side includingfixed applications. As an example, Dynamic Application 01 9620 may bethe most recently used application, and Dynamic Application 02 9618 maybe the second most recently used application, and so forth.

In particular embodiments, the top level of the hierarchy of theinteraction and transition model of the GUI may include only “faces,”and the next level of the hierarchy may include applications (or anyother features). As an example, the top level of the hierarchy mayinclude a home screen (e.g. the clock), and one or more faces, each facecorresponding to a different type of background, mode, or activity suchas a wallpaper (e.g. customizable by the user), weather information, acalendar, or daily activity information. Each of the faces may show thetime in addition to any other information displayed. Additionally, theface currently displayed may be selected by the user (e.g. via anysuitable input mechanism or type) or automatically change based oncontext (e.g. the activity of the user). The faces to the left of thehome screen may be contextual, and the faces to the right of the homescreen may be fixed. FIG. 97 illustrates an example layout of ahierarchy within the GUI in which the top level of the hierarchyincludes faces 9710-9770 (including clock face 9740) and the next levelof the hierarchy includes applications 9715-9775.

In particular embodiments, an input from a user of the device or aninput from another input source (e.g. via any of the variety of inputmechanisms or types including the outer ring, touch-sensitiveinterfaces, gestures, speech, or sensors), or a context of use of thedevice may cause a transition within the GUI from a screen at one levelof the hierarchy of the interaction and transition model of the GUI to ascreen at another level of the hierarchy. For example, a selection eventor input by the user (e.g. a touch or tap of the display, voice input,eye gazing, clicking or pressing of the outer ring, squeezing of theouter ring, any suitable gestures, internal muscular motion detected bysensors, or other sensor input) may cause a transition within the GUIfrom a top-level screen to a screen nested one level deeper in thehierarchy. If, for example, the current screen is a top-level screenassociated with an application, a selection event (e.g. pressing thering) selects the application and causes the GUI to transition to ascreen nested one layer deeper. This second screen may, for example,allow for interaction with a feature of the selected application andmay, in particular embodiments, correspond to a main function of theselected application. There may be multiple screens at this second,nested layer, and each of these screens may correspond to differentfunctions or features of the selected application. Similarly, a “back”selection input or event by the user (e.g. a double pressing of theouter ring or a touch gesture in a particular part of the display) maycause a transition within the GUI from one screen (e.g. a feature of aparticular application) to another screen that is one level higher inthe hierarchy (e.g. the top-level application screen).

FIG. 98A illustrates an example of the operation of the interaction andtransition model with respect to a function or a mode 9805 of aparticular application of the device and the use or application of thefunction 9810. As an example, if the application is a camera, thefunctions, modes, or other elements of the camera application mayinclude picture mode, video mode (e.g. with a live view), and turning onor off a flash. The various functions, modes, or other elements may beaccessed via transitions within a single layer of the model hierarchy.These intra-layer transitions may occur upon receiving or determining aparticular type of transition event or input from an input source suchas the user of the device (e.g. a rotation of the outer ringcounterclockwise or clockwise), or upon determining a particular contextof use of the device. In particular embodiments, a transition eventinput may also include, e.g., a touch or tap of the display, voiceinput, eye gazing, clicking or pressing of the outer ring, squeezing ofthe outer ring, any suitable gesture, internal muscular motion detectedby sensors, or other sensor input. To select and use a function, mode,or other element of the application, the user may provide a particulartype of selection event or input (e.g. a tap or touch of the display, apress or click of the outer ring, a particular gesture, or sensorinput), causing an inter-layer transition within the GUI to a deeperlayer of the hierarchy. As an example, to take a video, the user may tapa screen associated with the video mode feature of the cameraapplication. Once in this deeper layer of the hierarchy, taking a video,the user may cause the GUI to transition between different options inthat layer, if available (e.g. options related to video mode). Inparticular embodiments, the user may select one of the options in thedeeper layer, causing the GUI to transition to an even deeper layer. Asan example, once recording video in video mode, the user may again tapthe display to transition the GUI to a deeper layer, which in this casemay include the option to stop recording video. Additionally, the usermay return to a higher layer of the hierarchy by providing a particulartype of selection event or input (e.g. a “back” input, describedherein). As an example, once recording video in video mode, the user maytouch a particular “back” portion of the display, causing videorecording to be canceled and causing the GUI to transition to the screenassociated with the video mode feature of the camera application (e.g.in the features layer of the hierarchy). The interaction and transitionmodel hierarchy of the GUI may have any number of layers and any numberof elements (e.g. functions or content) within a single layer. FIG. 98Billustrates an example of the operation of the interaction andtransition model with respect to content 9815 on the device. In thisexample model, content may behave similarly to an application, exceptthat if the user selects the content 9815 (e.g. a photo) and the GUItransitions to a deeper layer in the hierarchy, the first option 9820 ina menu of options related to the content may be shown (e.g. options suchas deleting the photo or sharing the photo). FIG. 98C illustrates anexample of the operation of the interaction and transition model withrespect to a control 9825 on the device. A control element may functionlike a knob, in that it may modify a value over a range of possiblevalues. User input to the device (e.g. rotating the outer ring to theright or left) may modify the value or state 9830 associated with thecontrol element 9825. The value modified by a control element may besubstantially continuous in nature (e.g. the zoom level of a camera, orthe volume level of a television) or may be substantially discrete innature (e.g. the channel of a television). In particular embodiments, incases where the value modified by a control is discrete in nature, aparticular user input (e.g. pressing the outer ring) may “commit” theselection of the value. FIG. 98D illustrates an example of the operationof the interaction and transition model with respect to an application9835 on the device and a main function 9840 of the application. As anexample, each mode or function of the device (e.g. camera or augmentedreality functions) may be an application on the device. Transitionswithin a single layer (e.g. performed upon receiving a particular userinput such as a rotation of the outer ring) allow the user to changeapplications, modes, or functions of the device. Transitions betweenlayers (e.g. performed upon receiving a particular user input such as atap on the display) allow the user to enter deeper layers (or exitdeeper layers) of the hierarchy associated with the selectedapplication, mode, or function.

FIG. 98E illustrates an example of the operation of the interaction andtransition model with respect to an action 9845 (e.g. within anapplication) on the device. As an example, within the cameraapplication, a captured image may be selected, and one or more actionsmay be available for the selected image, such as deleting the image,sharing the image on FACEBOOK, sharing the image on TWITTER, or sendingan e-mail with the image. In this example, GUI transitions within the“action” layer (e.g. performed upon receiving a particular user inputsuch as a rotation of the outer ring) allow the user to view differentactions to take. Transitions between layers (e.g. performed uponreceiving a particular user input such as a tap on the display) allowthe user to enter deeper layers (or exit deeper layers) of the hierarchyassociated with the selected action. In this example, the deeper layerentered by selecting an action 9845 shows secondary information 9850 ora confirmation (e.g. that the application is sending the imageinformation to a selected sharing service). A confirmation 9855 (e.g.that the image has been sent) may also be shown in this deeper layer.The GUI may automatically transition back to a higher layer (e.g. theaction layer). There may, however, be a deeper layer of the hierarchyincluding the confirmation information, and this deeper layer may beentered by the GUI upon user input or automatically. FIG. 98Fillustrates an example of the operation of the interaction andtransition model with respect to an icon (e.g. an active icon 9860including a top-level on/off option) and the switching of the state ofthe icon 9865. As an example, a television communicatively paired withthe device may be indicated by an active icon, for example, a televisionscreen. In this example, GUI transitions within the device/applicationtop layer (e.g. performed upon receiving a particular user input such asa rotation of the outer ring) allow the user to view differentapplications, device, or other features. The television may appear in amenu in the GUI of the device even when the television is off, but thetelevision must be turned on before it may be used. If the user selectsthe television (e.g. by tapping on the display when the television iconis displayed by the GUI) when it is off 9860, the GUI may transition toa state in a deeper layer of the interaction and transition modelhierarchy in which the television is turned on 9865. When the televisionis turned on, the icon associated with the television (displayed, forexample, in the top layer of the model in the GUI) 9870 may change todirectly represent that the television has been turned on 9875, asillustrated in FIG. 98G. If the user again selects the television (nowon), the GUI may transition to an even deeper layer of the hierarchy inwhich functions or capabilities of the television (e.g. volume orchannel changing) are exposed. In particular embodiments, the option toturn the television off again may be the first menu item in this deeperlayer of the hierarchy, to enable quick access to the off function (e.g.in case the user has accidentally turned on the television). Inparticular embodiments, if the user selects the television when it isoff, the television may be turned on and the icon associated with thetelevision may change to directly represent that the television has beenturned on without the GUI transitioning to a different layer of thehierarchy or to a different user interface. The active television iconmay, therefore, directly indicate within the top level of the hierarchy(e.g. a main menu) the state of the paired television.

FIG. 99 illustrates an example of the interaction and transition modelhierarchy of a GUI for an image capture application. In this example,the first screen 9902 arrived at after selection of the application (atscreen 9900) may correspond to a “live view” function of theapplication. Other fixed features of the image capture application,including video mode 9904, zoom 9906, or flash 9908, may be available tothe right of the home main function screen 9902 of the selectedapplication. Dynamically or contextually available features (e.g.captured images 9910) of the selected application may be available tothe left of the home main function screen. A selection event at thisfunctional layer of the hierarchy may cause a transition within the GUIto another nested layer even deeper within the hierarchy. If, forexample, the user selects the “zoom” function, the GUI may transition toa screen 9912 in which the user may control the zoom setting of a camerawith any suitable input (e.g. a rotation of the outer ring to the rightto increase zoom or a rotation of the outer ring to the left to decreasezoom). Similarly, the user may be able to control the state of differentfeatures (e.g. turning a flash feature on or off 9914, or switching froma picture mode to a video mode 9916), browse content (e.g. 9918-9922),enter a deeper layer of the hierarchy in which actions 9924-9930 may betaken, or enter yet another, even deeper layer of the hierarchy in whichconfirmations 9932-9938 are provided once an action is selected.

In particular embodiments, an interaction layout may structure aninteraction and transition model of a GUI of the device. An interactionlayout may be applied to any suitable interaction model and need not bedependent on any specific type of motion or animation within a GUI ofthe device, for example. Although specific examples of interactionlayouts are discussed below, any suitable interaction layout may be usedto structure an interaction and transition model.

As one example, a panning linear interaction layout may structure aninteraction and transition model of a GUI of the device. In apanning-linear-type GUI, elements or features within a layer may bearranged to the left and right of the currently displayed element orfeature. User input such as a rotation of the outer ring in a clockwiseor counterclockwise direction navigates within a single layer of themodel hierarchy. As an example, a rotation of the outer ring clockwiseone rotational increment may display the element or feature to the right(e.g. the next element), and a rotation counterclockwise one rotationalincrement may display the element or feature to the left (e.g. theprevious element). In particular embodiments, a fast rotation clockwiseor counterclockwise may cause the GUI to perform accelerated browsing.In such an embodiment, a single turn may cause the GUI to transitionthrough multiple elements or features, rather than a single element orfeature, as described herein. Different user input may navigate betweenlayers (e.g. either deeper layers or higher layers) in the modelhierarchy. As an example, if the user touches or taps thetouch-sensitive layer of the display, the GUI may transition one layerdeeper in the model hierarchy (e.g. confirming the user's selection orproviding options related to the selection). Any suitable input by theuser may cause the GUI to transition between layers in the modelhierarchy, either in place of or in addition to touch- or tap-basedinput.

As another example, if the user presses a particular region of thetouch-sensitive layer of the display (e.g. designated as a “back”button), or if the user double-taps the touch-sensitive layer of thedisplay, the GUI may transition one layer higher in the model hierarchy(e.g. to the previous layer). If, for example, the user performs a longpress of the display or screen, the GUI may transition back to the homescreen (e.g. a clock). Without additional user input, the GUI may alsotransition back to the home screen after a pre-determined period of time(e.g. a timeout period). As described herein, as a user begins, forexample, to rotate the outer ring in a clockwise or counterclockwisefashion, the GUI transitions within the same layer, and the next userinterface element or feature (e.g. a breadcrumb icon in the same layer)to the right or left, respectively, may begin to appear while thecurrent user interface element or feature may begin to disappear.

FIG. 100A illustrates an example of the panning linear interactionlayout. In this example, GUI elements 10001, 10002, 10003, and 10004 arein the same layer of the interaction and transition model hierarchy ofthe panning-linear-type GUI. GUI elements 10002A, 10002B, and 10002C areelements in a second, deeper layer of the hierarchy and are sub-elementsof element 10002. As an example, the first layer may include devicespaired with the device—element 10001 may represent an automobile,element 10002 may represent a television, element 10003 may represent amobile phone, element 10004 may represent a home thermostat. Element10002A may be a volume control element for the television, element10002B may be a channel control element for the television, and element10002C may be a picture control element for the television. As yetanother example, the GUI may transition one layer deeper in thehierarchy if the user clicks the ring (e.g. presses down on the ringonce), and then sub-elements in the deeper layer may be panned byrotating the ring. Alternatively, the user may pan the sub-elements inthe deeper layer by rotating the ring while simultaneously pressing downon the ring. The device may include a switch to select how the userinput is used to navigate between layers.

As another example, a panning radial (or panning circular) interactionlayout may structure an interaction and transition model of a GUI of thedevice. In a panning-radial-type GUI, elements or features in a layermay be arranged above and below the currently displayed element orfeature. User input such as a rotation of the outer ring in a clockwiseor counterclockwise direction navigates between layers of the modelhierarchy. As an example, a rotation of the outer ring clockwise oneincrement may cause the GUI to transition one layer deeper in the modelhierarchy (e.g. entering a particular application's layer or confirmingselection of the application), and a rotation counterclockwise oneincrement may cause the GUI to transition one layer higher in the modelhierarchy (e.g. exiting a particular application's layer to the previouslayer). In particular embodiments, a fast rotation clockwise orcounterclockwise may cause the GUI to perform accelerated browsing, asdescribed herein. In such an embodiment, a single rotational incrementmay cause the GUI to transition through multiple layers of thehierarchy, rather than a single layer. Different user input may navigatewithin a single layer in the model hierarchy. As an example, if the usertouches or taps the touch-sensitive layer of the display, the GUI maytransition to the next element or feature (e.g. the element below thecurrently displayed element). As another example, if the user presses aparticular region of the touch-sensitive layer of the display (e.g.designated as a “back” button), or if the user double-taps thetouch-sensitive layer of the display, the GUI may transition to aprevious element or feature (e.g. the element above the currentlydisplayed element). If, for example, the user performs a long press ofthe display or screen, the GUI may transition back to the home screen(e.g. a clock). Without additional user input, the GUI may alsotransition back to the home screen after a pre-determined period of time(e.g. a timeout period). As described herein, as a user begins, forexample, to rotate the outer ring in a clockwise or counterclockwisefashion, the GUI transitions to a different layer, and the next userinterface element or feature (e.g. in a different layer) may begin toappear while the current user interface element or feature may begin todisappear. FIG. 100B illustrates an example of the panning radialinteraction layout. In this example, GUI elements 10001, 10002, 10003,and 10004 are in the same layer of the interaction and transition modelhierarchy of the panning-radial-type GUI. GUI elements 10002A, 10002B,and 10002C are elements in a second, deeper layer of the hierarchy andare sub-elements of element 10002. As before, the first layer mayinclude devices paired with the device—element 10001 may represent anautomobile, element 10002 may represent a television, element 10003 mayrepresent a mobile phone, element 10004 may represent a home thermostat.Element 10002A may be a volume control element for the television,element 10002B may be a channel control element for the television, andelement 10002C may be a picture control element for the television.

As yet another example, an accordion-type interaction layout maystructure an interaction and transition model of a GUI of the device. Inan accordion-type GUI, elements or features of multiple layers may bearranged in a circular list structure. For example, rotating within thelist structure (e.g. by rotating the outer ring) in a first directionpast a screen associated with the last element or feature in thatdirection (e.g. the last fixed application of the device) may cause theGUI to transition to a screen associated with the last element orfeature in a second direction (e.g. the least-recently used contextualapplication of the device). Continuing to rotate in the first directionmay cause the GUI to transition through screens associated withcontextual applications in “reverse” order (e.g. from least-recentlyused to most-recently used). Similarly, rotating in the second directionpast the screen of the least-recently used contextual application maycause the GUI to transition to the screen associated with the last fixedapplication, and continuing to rotate in the second direction may causethe GUI to transition through the screens of the fixed applications inreverse order (e.g. from the last fixed application to the first,adjacent to the home screen). In an accordion-type GUI, the element orfeature currently displayed may be “expanded” (e.g. if selected by theuser) such that its sub-elements or sub-features may become part of thesingle-layer list structure. In particular embodiments, an element orfeature with sub-elements may indicate (when displayed) that it hassub-elements through, for example, visible edges of the sub-elements.User input such as a rotation of the outer ring in a clockwise orcounterclockwise direction navigates within a single layer of the model,which may include elements or features, as well as sub-elements orsub-features of a selected element or feature. As an example, a rotationof the outer ring clockwise one increment may display the element orfeature to the right (e.g. the next element), and a rotationcounterclockwise one increment may display the element or feature to theleft (e.g. the previous element). In particular embodiments, a fastrotation clockwise or counterclockwise may cause the GUI to performaccelerated browsing. In such an embodiment, a single rotationalincrement may cause the GUI to transition through multiple elements orfeatures, rather than a single element or feature. Different user inputmay cause the selection and expansion of an element or feature in themodel. As an example, if the user touches or taps the touch-sensitivelayer of the display, the GUI may expand the displayed feature orelement within the existing layer and transition to a sub-element orsub-feature. As another example, if the user presses a particular regionof the touch-sensitive layer of the display (e.g. designated as a “back”button), or if the user double-taps the touch-sensitive layer of thedisplay, the GUI may collapse the expanded sub-elements or sub-featuresand transition to an element or feature in the list. If, for example,the user performs a long press of the display or screen, the GUI maytransition back to the home screen (e.g. a clock). Without additionaluser input, the GUI may also transition back to the home screen after apre-determined period of time (e.g. a timeout period). As describedherein, as a user begins, for example, to rotate the outer ring in aclockwise or counterclockwise fashion, the GUI transitions within thesame layer, and the next user interface element or feature (e.g. abreadcrumb icon in the same layer) to the right or left, respectively,may begin to appear while the current user interface element or featuremay begin to disappear. FIG. 100C illustrates an example of theaccordion-type interaction layout. In this example, GUI elements 10001,10002, 10003, and 10004 are in the same layer of the interaction andtransition model of the accordion-type GUI. Because element 10002 hasbeen selected by the user, GUI sub-elements 10002A, 10002B, and 10002Care expanded and also included in the list structure in the same layerof the model. Thus, the GUI may transition from sub-element 10002C toeither sub-element 10002B or directly to element 10003. If, however, theuser desires to collapse the sub-elements (e.g. through a “back” inputsuch as tapping the screen associated with element 10002 again), thenthe list structure will only include GUI elements 10001, 10002, 10003,and 10004 again.

In particular embodiments, the GUI may navigate to a home screen basedon input received by a user of the device. The user input may include,for example, pressing and holding (e.g. a long press) thetouch-sensitive layer, pressing and holding the display, pressing (e.g.clicking) and holding the outer ring, squeezing and holding the outerring, covering the face (e.g. the display) of the device, covering aparticular sensor of the device, turning the face of the device in adownward direction, pressing a software button (discussed herein),pressing a hardware button on the device, or shaking the device (or anyother suitable gesture). Any of these inputs or any variation of theseinputs (including, for example, shorter durations) may be used as userinputs to go “back” within an interaction and transition model. FIGS.101A-101B illustrate examples of a “back” software button layout in theGUI. In FIG. 101A, receiving user touch input in the bottom portion10110 of the display causes the GUI to confirm a selection or transitionone layer deeper in the model hierarchy. Receiving user touch input inthe top portion 10120 of the display causes the GUI to transition “back”or one layer higher in the model hierarchy. FIG. 101B illustrates asimilar layout, with the “back” region 10130 including a breadcrumb icon10135 to indicate to the user where navigating “back” will transition.In particular embodiments (e.g. when the touch-sensitive layer isoperable to determine precise x-y coordinates of a touch), any region ofthe display may be designated as a “back” region, a “confirm/select”region, or any other suitable functional region.

In particular embodiments, the GUI of the device may display particulartypes of content including, for example, lists. FIG. 102A illustrates anexample of the GUI displaying a vertical list of items. An input fromthe user (e.g. any suitable input mechanism or type) may cause aselection frame 10210 of the GUI to move through elements of thevertical list. As an example, if the user rotates right in a clockwisedirection, the selection frame 10210 may move from the top of thevertical list toward the bottom of the vertical list. Each rotationalincrement of the outer ring (e.g. if the outer ring moves in discreteincrements), causes the selection frame 10210 to move one item withinthe list. In the example of FIG. 102A, as the user rotates the ringclockwise, the displayed items of the list remain constant, and theselection frame 10210 moves downward through items of the list. In otherembodiments, the selection frame may remain constant (e.g. in the centerof the display), and items of the list may move upward or downward (e.g.one item at a time), depending on the direction of the ring's rotation.FIG. 102B illustrates an example of the GUI displaying a horizontal listof items. An input from the user (e.g. any suitable input mechanism ortype) may cause a selection frame 10210 of the GUI to move throughelements of the horizontal list. As an example, if the user rotatesright in a clockwise direction, the selection frame 10210 may move fromthe left of the horizontal list toward the right of the horizontal list.Each rotational increment of the outer ring (e.g. if the outer ringmoves in discrete increments), causes the selection frame 10210 to moveone item within the list. In the example of FIG. 102B, as the userrotates the ring clockwise, the selection frame 10210 remains constantin the center of the display, and items of the list move toward the left(e.g. one item at a time) in response to the clockwise rotation. Inother embodiments, the displayed items of the list remain constant, andthe selection frame moves left or right through items of the list,depending on the direction of rotation of the outer ring.

In particular embodiments, the GUI of the device may display verticallyor horizontally continuous (or substantially continuous) contentincluding, for example, charts or text. In particular embodiments, aninput from the user (e.g. any suitable input mechanism or type) maycause a selection indicator of the GUI to move through the continuouscontent. In other embodiments, an input from the user may cause thecontent to move into and out of the display in a horizontal direction,vertical direction, or any other direction mapped to the user's input(and the selection indicator, if present, may remain in a constantposition). In the example of FIG. 102C, a temperature chart isdisplayed. As the user rotates the outer ring in a clockwise fashion,the selection indicator 10220 remains in the center of the display, andthe content moves into the display from the right and out of the displaytoward the left. In the example of FIG. 102D, a portion of a largerpiece of text 10230 is displayed. As the user rotates the outer ring ina clockwise fashion, additional text enters the display from the bottomand exits the display toward the top. FIGS. 103A-103D illustrate anexample calendar application displayed in GUI of the device. In FIG.103A, a user may click or press the outer ring (indicated by arrow10305), causing the GUI to display a circular menu 10310 with options“Go Up,” “Weekly” (the default setting), “Monthly,” and “Daily.” In FIG.103C, the user may again click or press the outer ring (indicated byarrow 10305), confirming selection of “Weekly” and causing the GUI todisplay the weekly view 10320 of the user's calendar.

In particular embodiments, the GUI may display content that is of a sizelarger than the display. In such embodiments, the GUI may scale or crop(or otherwise shrink or fit) the content so that all of the content maybe displayed within the display at one time. In other embodiments, theGUI does not alter the size of the content, and instead provides theability for the user to pan through the content one portion at a time,for example using scrolling (described herein).

In particular embodiments, the device includes the circular display, andthe GUI includes circular navigation and menu layouts. This disclosurecontemplates any shape for the display, however, and any suitablenavigation or menu layout for the GUI. The menu layout may provide auser a visual indication of where the user is located within aninteraction and transition model hierarchy of the GUI, for example. Themenu layout may also provide visual indicators that allow the user todifferentiate between different types of menu items, as well as show anoverall view of menu options. Additionally, the menu may be displayedover any suitable background or content of the device.

FIG. 104 illustrates an example circular menu layout in which eachsegment 10410 represents one item or option in the menu and visual gapssuch as 10420 separate the items from one another. The default orcurrently selected item 10430 is on the top of the visual display (butmay be anywhere on the display), and may remain at the top of thedisplay as the user orients the device display in different ways duringuse. FIGS. 105A-105B illustrate an example of browsing the items in acircular menu. The user may provide input such as a clockwise rotationof the outer ring, and in response to this user input, the next item inthe menu 10520 (e.g. to the right of the currently selected item 10510)may be highlighted for selection. The content in the center of thedisplay 10530 may automatically change to reflect the user's rotationinput or may, in particular embodiments, change only after the userprovides another input (e.g. pressing or clicking the outer ring oncethe desired menu item is highlighted). FIGS. 105C-105D illustrate anexample of browsing a circular menu by rotating the outer ring, causingthe next item in the menu 10550 (e.g. clockwise or to the right of thecurrently selected item 10540) to be highlighted for selection. In thisexample, the user's input also causes the rotation of a central“pointer” 10560 that points at the highlighted menu segmentcorresponding to the currently-selected menu item. In this example, thecontent in the center of the display automatically changes to reflectthe user's rotation.

FIGS. 106A-106C each illustrate different alignments and arrangements ofa circular menu layout for the GUI of the device. The circular menu may,for example, be displayed directly on the border of the display (asshown in FIG. 106A) or may be shown further inside the display, or as anoverlay over a background of the device (shown in FIGS. 106B-106C).FIGS. 107A-107C illustrate other forms and alignments of a circular menulayout for the GUI of the device. As examples, the menu may consist ofline segments (of various possible sizes) arranged in a circle 10710,line segments arranged in a semicircle 10720, or dots arranged in acircle or semi-circle, 10730 or 10740. In particular embodiments, thevisual indicator of the currently selected or default menu item 10732may remain at the top center of the display, and the visual indicatorsof items in the menu 10734 may shift left or right based on user input(FIG. 107C). In other embodiments, the visual indicator of the currentlyselected or default item 10732 may move through the indicators of theitems of the menu, which remain fixed in position (FIG. 107B). Inparticular embodiments, instead of segments or dots, the visualindicators of items in the menu may be icons (e.g. breadcrumb icons)associated with the menu items. FIG. 108 illustrates that the menulayout need not be circular and may be any suitable layout, including alayout in which indicators of menu items 10810 are scattered throughoutthe display. With user input (e.g. a rotation of the outer ring),different items may be selected according to their position in the menulayout. As an example, if the user rotates in a clockwise manner, thenext menu item 10820 in a clockwise direction may be selected.

FIGS. 109A-109C illustrate different menu layouts with respect to menuitems to the “left” and to the “right” (e.g. in the interaction andtransition model hierarchy) of the currently selected or displayed menuitem 10915. In FIG. 109A, all menu items 10910 are equally distributedon the circular menu around the display. In FIG. 109B, the menu includesa gap which indicates a differentiation of items 10910 to the left anditems to the right of the currently-displayed or selected menu item10915 (e.g. in accordance with the interaction and transition modeldescribed herein). FIG. 109C illustrates an example in which there aremore items 10910 to the left than to the right of the currently-selectedor displayed item 10915, so that the left-hand segments of the circularmenu are adjusted in size to accommodate the number of items availablefor selection. In the case of a large number of menu items (e.g. beyonda particular threshold such as 40 captured images), the segments of thecircular menu may disappear, and the visual indicator presented to theuser may be a scroll bar 11020 that allows the user to circularly scrollthrough the various menu items, as illustrated in FIG. 110A. In otherembodiments, a similar scrollbar-type visual indicator 11020 may allowthe user of the device to manipulate an absolute or fixed value (e.g. acamera zoom level) over a fixed range of values 11030, as illustrated inFIG. 110B. In yet other embodiments, the length of a scrollbar-typevisual indicator may show the user the level of a certain value. Forexample, if the user is controlling the volume of a television using theouter ring of the device, as the user turns the ring (e.g. clockwise) toincrease the volume level, the visual indicator 11120 will grow longer,until it encircles or nearly encircles the entire display, asillustrated in FIGS. 111A-111C.

In particular embodiments, the GUI may display both an item of referenceor background content as well as an indication of an available action orfunction to be performed with respect to the reference or backgroundcontent. FIG. 112 illustrates example layouts within the GUI ofreference content and contextual overlay actions or functions. Differenttypes of layouts (e.g. including those illustrated) may be selectedbased on the different types of reference or background contentpresented, for example, to minimize obscuring the reference orbackground content. For example, if the reference or background contentis a picture of a person, an overlay that does not obscure the center ofthe photo may be selected. In particular embodiments, the perceptualbrightness of the pixels of the reference or background content (e.g.behind the overlay) may be determined on a pixel-by-pixel basis. Incases where the contrast between the contextual overlay and thereference or background content (e.g. an image) is too low (e.g. basedon a pre-determined threshold), a blurred drop shadow that pushes theunderlying colors in the opposite direction may be used. An examplealgorithm may include determining the pixels under the overlay, reducingtheir saturation, taking the inverse of the visual brightness (e.g. suchthat colors remain the same but the brightness is selected to producecontrast), blur, and create a composite between the underlying referenceor background content and the overlay. FIGS. 113A-113C illustrateexamples 11310-11350, of contextual overlays composed with background orreference content (here, images captured by a camera of the device). Asillustrated, the contextual overlay may allow the user to performactions or functions (e.g. deleting an image 11130 or sharing an image11325, searching for coffee 11330, searching for restaurants 11340, ormaking a location a “favorite” location 11350), provide confirmation tothe user (e.g. that an image has been shared 11320), or provide anyother type of information to the user. In particular embodiments,contextual overlays may be used anywhere within a menu layout of a GUIexcept for the top level of the interaction and transition modelhierarchy.

In particular embodiments, icons displayed in the GUI of device mayoptimize the energy or battery usage of the device. As an example, anicon may include primarily black background with the icon itself beingcomposed of thin white strokes. This may allow for the amount of whitecolor on the display screen to be very low, allowing for reduced energyconsumption of the display while the GUI is used. The icons displayed inGUI may also include real-time notifications. For example, a mobilephone icon may include a notification with the number of new voicemails,an e-mail icon may include a notification with the number of newe-mails, a chat icon may include a notification with the number of newchat messages, and a telephone icon may include a notification with thenumber of missed calls. In particular embodiments, the GUI of the deviceonly displays colors other than black and white for user-generatedcontent (e.g. pictures, files, contacts, notifications, or schedules).Other information, including menu items, may be displayed in black andwhite.

In particular embodiments, as the GUI transitions from one element (e.g.feature, content item, or icon) to another (e.g. upon receiving inputfrom a user), the GUI may display visual transition effects. Thesetransition effects may depend, for example, on the type of inputreceived from a user of device. As an example, a single touch on thedisplay may trigger particular transition effects, while a rotation ofthe outer ring may trigger a different (potentially overlapping) set oftransition effects.

In particular embodiments, a user's touch input on the touch-sensitivelayer may trigger transition effects including center-orientedexpansion, directional sliding, and scaling in or out. FIG. 114Aillustrates center-oriented mode or function expansion or scaling up.FIG. 114B illustrates center-oriented mode or function collapsing orscaling down. FIG. 115A illustrates center-oriented scaling up of anicon. FIG. 115B illustrates center-oriented scaling down of an icon.FIG. 116A illustrates an example of center-oriented icon scaling up witha twisting motion. FIG. 116B illustrates an example of center-orientedicon scaling down with a twisting motion. FIG. 117A illustrates anexample of center-oriented unfolding and expansion outward of an icon.FIG. 117B illustrates an example of center-oriented folding andcollapsing inward of an icon. FIG. 118A illustrates an example of textvertically sliding into the display, where the text is revealed byunmasking. FIG. 118B illustrates an example of text horizontally slidingin from the left to the right of the display. FIG. 118C illustrates anexample of text horizontally sliding in from the left to the right ofthe display within a masked region (e.g. a contextual overlay). FIG.119A illustrates a horizontal slide transition from right to left forcontent or an icon. FIG. 119B illustrates a horizontal slide transitionfrom right to left, with fading effects; the icon or content exiting thescreen fades out gradually once it reaches the screen's border, and theicon or content entering the screen fades in gradually as it crosses thescreen's border. FIG. 119C illustrates an example of a horizontal slidetransition from right to left with scaling effects; the content or iconexiting the screen is shrunk down, and the content or icon entering thescreen is scaled up to full size.

In particular embodiments, a user's rotation of the outer ring maytrigger visual transition effects including zooming, directionalsliding, blurring, masking, page folding, rotational movement, andaccelerated motion. FIG. 120A illustrates an example of a transition inresponse to a low-acceleration rotation of the outer ring. In thisexample, a single rotational increment may correspond to a single item,such that one turn (e.g. rotational increment) counterclockwise causesthe next element (e.g. icon or content item) to enter the screen fromthe left toward the right, and no scaling of elements occurs. FIGS.120B-120C together illustrate an example of a transition in response toa high-acceleration rotation of the outer ring. In this example, asingle turn (e.g. rotational increment) counterclockwise causes the GUIto pan quickly through multiple elements (which may scale down in size,enter the screen from the left, and exit the screen from the right)until the user stops turning the ring. When the user stops turning theouter ring, the element may scale up to normal size, and a single iconor content item may fill the display. FIG. 121A illustrates an exampleof a transition within the GUI in which content is zoomed-in in responseto rotation of the outer ring. FIG. 121B illustrates an example of atransition within the GUI in which a first screen 1 “folds over” in ananimation, resulting in a second screen 2 (e.g. for the next feature orcontent item) being displayed to the user.

In particular embodiments, the GUI of the device may include a physicalmodel that takes into account motion of the user and produces visualfeedback reflecting the user's movements. As an example, once there isactivation input (e.g. in the form of a particular gesture) by the user,the user's motion may be continuously tracked through input from one ormore of the sensors of the device. The visual feedback may reflect theuser's motion in the user interface, while the underlying content staysstill, so that gestures may be registered and parallax may be used todistinguish between UI features or controls and underlying content. Inparticular embodiments, the physical model may include a generalizedspring model with damping. In such a model, items may be arranged inlayers. Deeper layer may have a “stiffer” spring in the physical modelholding items in place. This may cause bottom layers of the userinterface to move slightly when the device is moved, while top layersmay move more, creating a sense of parallax. Additionally, the springmodel may include damping, which causes motion to lag, creating a morefluid, smooth motion. FIG. 122 illustrate an example of using a physicalmodel in the GUI. The user wears the device 100 on her arm. Once theuser moves her arm in a downward fashion, the icon 12210 displayed onthe screen (e.g. a light bulb) moves in a manner reflecting the user'smovement. The underlying content (e.g. the background image) on thescreen does not move, however. This type of floating icon or menu itemmay, for example, be helpful when the display is of a size that does notallow for many icons or menu items to be displayed simultaneously due tovisual crowding. Additionally, this type of floating behavior may alsobe used with notification means for presenting an event to the user.

In particular embodiments, the GUI of the device may include faces asdefault screens or wallpapers for the device, and these faces may bepart of an interaction and transition model hierarchy (e.g. in the toplayer of the hierarchy or as a home screen). As described herein, thesefaces may be changeable applications or modes that may automaticallyrespond contextually to a user's activity. As an example, the faces maychange depending on the user's environment, needs, taste, location,activity, sensor data, gestures, or schedule. The availability of a face(or the transition in the GUI from one face to another) may bedetermined based on contextual information. As an example, if the userhas an upcoming event scheduled in her calendar, the face of the devicemay change to a calendar face that displays the upcoming eventinformation to the user. As another example, if the user is determinedto be in the vicinity of her home (e.g. based on GPS data), the face ofthe device may change to a face associated with a home-automationapplication. As yet another example, if the user is determined (e.g.based on various biometric sensors such as heart rate or arousalsensors, or based on accelerometers) to be moving vigorously, the faceof the device may change to a fitness mode, showing the user's measuredpulse, calories burned, time elapsed since the activity (e.g. a run)began, and the time. Any suitable sensor data (e.g. from sensorsincluding biometric sensors, focus sensors, or sensors which maydetermine a user's hand position while driving a car) may be used todetermine a context and appropriate face to display to the user. Theuser's historical usage of the device (e.g. a particular time of daywhen the user has used a fitness application, such as in a fitnessclass) may also determine which face is displayed on the device. As anexample, the device may anticipate the user's need for the fitness modeat the particular time of day when the user tends to exercise.Contextual faces may also be associated with the suppression ofnotifications (e.g. if the user is determined to be driving or if thedevice is not being worn) or a change in how notifications are expressed(e.g. visually, or audibly). In particular embodiments, the faces of thedevice need not be associated with any application on the device and maybe wallpapers or backgrounds on the display of the device. Faces may bededicated to specific channels of information (e.g. calendar feeds,health or activity feeds, notifications, weather feeds, or news). As anexample, a severe weather notification or alert (received, e.g., from aweather feed) may cause the weather face to be displayed on the displayalong with the notification. Faces may display the time (e.g. in analogor digital format) regardless of the type of face. The faces may becustomizable by the user. The user's customizations or tastes may beinput explicitly by the user (e.g. to management software on the deviceor a paired device) or learned directly by the device (e.g. using sensorand usage data to create a model over time). FIG. 123 illustratesexample faces, including an analog watch 12310, an analog watch with acircular menu layout 12320, a health-mode face 12330, and a weather face12340. FIG. 124 illustrates an example set of faces 12410-12440 for thedevice in which calendar and appointment information is displayed.

In particular embodiments, the device may be worn on a limb of a user(without obscuring the user's face and without requiring the user tohold the device) and may include augmented reality (AR) functionality.This AR functionality may be based on the use of body motion for aiminga camera of the device, which may allow for aiming with higher accuracydue to a user's sense of proprioception. This type of system may allowthe user of the device to view an object in the real world at the sametime that the user views a version of the object (e.g. captured by acamera of the device) on the display. An example of this AR capabilityis illustrated in FIG. 16. Such an AR system may allow for “see-through”capability using an aligned camera and sensor on opposite sides of auser's limb. Various AR applications may be enabled by this type ofarrangement, described herein. In particular embodiments, applicationsmay be designed specifically for the device to allow for immediate,opportunistic use. Additionally, a delegation model may be provided onthe device, allowing for the use of external resources to improve thebreadth of applications available to run on the device while incurringless (or no) penalty in terms of processing requirements or energy use.In particular embodiments, the device may control or be controlled byother devices (e.g. nearby devices discovered via a network andcommunicatively paired with the device). This type of control may beachieved via proximity, gestures, or traditional interfaces. Pairing maybe achieved using a variety of technologies including a camera of thedevice, discussed in further detail herein.

FIG. 125 illustrates an example of an automatic camera activationdecision flow for the device. In particular embodiments, whether thecamera is enabled and whether automatic activation of the camera (e.g.for object recognition) is enabled may depend on the application or modethe device is currently in. In particular embodiments, automatic cameraactivation may be enabled on the device 12510. If this feature isenabled (determined at step 12520) and if there is sufficient CPUcapacity and power available on the device (e.g. to calculate featuresof interest from an image, determined at step 12530), then a camera ofthe device (e.g. an outward-facing camera) may automatically capture,process, or display 12560 one or more images if the camera is heldsteadily in an aiming position by the user for a pre-determined amountof time (e.g. as detected by an inertial measurement unit on thewearable device or as calculated by the blurring of the image,determined at step 12540). In other embodiments, the camera may beactivated and searching for images at all times. In yet otherembodiments, the camera may capture an image and perform featurerecognition only if the user manually triggers image capture (e.g.pressing or clicking the outer ring, or tapping the display, determinedat step 12550). In particular embodiments, when the camera is activated(by any suitable method), augmented reality (AR) functionality may beenabled. The AR functionality may be automatically enabled (depending,e.g., on CPU capacity and power available on the device). In otherembodiments, AR functionality may be explicitly enabled by the user viaany suitable input by the user. The user may, for example, provide touchinput on the display to enable AR functionality. As an example, a usermay capture an object such as a bird (e.g. by pointing a camera of thedevice at the bird), and the user may touch the image of the bird asdisplayed on the display. This action may enable the AR functions of thedevice, causing, for example, the device to recognize the bird as anobject and return information about the bird to the user. In otherembodiments, as described herein, the user may perform one or moregestures to enable AR functionality, as well as to perform tasks usingAR functionality (e.g. using a “virtual” keyboard by performing typinggestures in view of a camera of the device).

In particular embodiments, if the device does not have the capability tocalculate features of interest itself, the device may capture an image,transfer the image to a communicatively coupled device (e.g. a nearbydevice such as a phone or personal computer) or to an Internet-basedservice, where the features of interest may be calculated remotely. Oncethe features of interest are determined, an Internet-based service orlocal data catalog may be consulted for additional information about arecognized object. If information is found, the relevant data may bedisplayed to the user on the device along with the recognized feature.

The device may, in particular embodiments, have a small form factor andbe constrained in terms of available memory, processing, and energy. Adelegation model may allow the device to delegate portions of one ormore processing tasks (e.g. tasks related to AR functionality) to nearbydevices (e.g. phone or personal computer) or to network- orInternet-based services, for example. As an example, for delegabletasks, the application requiring the task provides the system (e.g. akernel of an operating system of the device) with characteristics or aprofile of the task, including the task's latency sensitivity,processing requirements, and network payload size. This may be done foreach delegable subtask of the overall delegable task. Since tasks areoften pipelined, contiguous chunks of the task pipeline may bedelegated. The system may, in particular embodiments, take measurementsof or build a model of one or more characteristics of the device.Characteristics of the device may include static properties of thedevice, e.g. properties of hardware components of the device includingtotal memory installed, maximum CPU speed, maximum battery energy, ormaximum bandwidth of a network interface. Characteristics of the devicemay also include dynamic properties of the device, e.g. operatingproperties of the device including available memory, current CPUcapacity, available energy, current network connectivity, availabilityof network-based services, a tally of average user behavior among one ormore users, or a predicted or expected processing time of a task (e.g.given a particular usage scenario). In particular embodiments, thedevice may have a model that incorporates previous and currentmeasurements of device characteristics to aid in determining futuredevice behavior. Based on the task characteristics or profile and thesemeasurements or models, as well as based on whether the task may beexecuted on the device, the system may delegate (or not delegate) one ormore portions of the task or task pipeline. For example, if theavailable memory on the device cannot support the processing of a task(e.g. playing a video), one or more portions of the task may bedelegated. As another example, if the CPU capacity of the device cannotsupport processing a task (e.g. if the CPU is running at capacity due toits existing load), one or more portions of the task may be delegated.As another example, if a battery level of the device is low and thebattery is not expected to provide energy to the device for as long asthe expected processing time of the task, one or more portions of thetask may be delegated. As another example, if the network connectivityof the device is low or non-existent, one or more portions of the taskmay not be delegated (e.g. if the device also has enough availablememory, CPU capacity, and energy). As another example, if one or morenetwork-based services are available to the device (e.g. cloud-basedservices for processing) and the device has suitable networkconnectivity (e.g. good available bandwidth), one or more portions ofthe task may be delegated. As another example, if a user of the devicetypically (e.g. historically) delegates the playing of videos, one ormore portions of the task of playing a video may be delegated. Asanother example, if a predicted processing time of the task (e.g.predicted based on a model incorporating previous and currentmeasurements of device characteristics) is beyond a certain threshold(e.g. several minutes), the task may be delegated. Any suitablecharacteristics of the device (e.g. static or dynamic properties) in anysuitable combination may be used to determine whether to delegate atask. Furthermore, any suitable characteristics of a task of the device(e.g. including a task profile or characteristics of the task includinglatency sensitivity, processing requirements, or network payload size)may be used to determine whether to delegate a task, either alone or inconjunction with device characteristics. Additionally, any model of thedevice (e.g. device behavior) may be used, either alone or inconjunction with device or task characteristics, may be used todetermine whether to delegate a task. In particular embodiments, devicespaired with the device may also include a delegation model, such thatthe paired device (e.g. a phone) performs the same steps, delegatingtasks based on its own models of energy, connectivity, runtimerequirements, and feasibility. The delegated task may be processed orrun to completion on the paired device (e.g. phone), and the results ofprocessing the delegated task may be returned to the device. Inparticular embodiments, the device may operate in standalone mode (e.g.without delegating any processing tasks) when it does not have anynetwork connectivity or when no paired devices are in range of thedevice. Once the device regains connectivity, or when a device is pairedwith the device, delegation of tasks may resume.

An example algorithm of a delegation model of the device is illustratedin FIG. 126. In this example, a delegable task process begins on thedevice (12610). The system of the device performs a power use analysisand prediction (12620) (based, e.g., on the user's historical energyusage 12630 and the expected time until a charge of the device 12640).Based on this, the system determines at step 12650 whether there issufficient charge remaining for the required uptime of the delegabletask. If sufficient charge remains, the system of the device mayincrease the power usage 12660 and process the delegable task on thedevice itself 12670. If, however, the device does not have sufficientcharge for the required uptime, the device may query a paired device(e.g. a phone) 12680 to determine the energy status of the paired device(12690). If, in the example of a phone, there is sufficient chargeremaining on the phone for the required uptime, the task may beprocessed on the phone 12694. If, however, there is not sufficientcharge on the phone, the system may determine at step 12692 if thedevice has connectivity to an Internet-based (e.g. cloud) or othernetwork-based service. If not, the device may delegate the process tothe phone 12694. If there is connectivity, the device may delegate theprocess to the cloud 12696, where the task is processed and the resultslater returned to the device. In particular embodiments, delegable tasksmay be delegated by the device in a divided fashion to one or morepaired devices (e.g. mobile phones or personal computers) ornetwork/Internet services. That is, delegable sub-tasks of a delegabletask or process may be delegated by the device to different locations.

It is contemplated by this disclosure that a delegation model for aparticular the device (or for a family or range of devices) may bedynamic or contextual. As an example, a delegation model may take intoaccount available memory, CPU capacity, and available energy of aparticular the device (or a family of devices), factors which may allchange over time. The delegation model may also take into account theavailability of network- or cloud-based services (and the capacity ofeach), as well as network connectivity (e.g. bandwidth and latency),which may also change over time. For example, with reference to FIG.127, according to a first delegation model 12710 (which may, e.g., beapplicable for devices manufactured in the next year), most processingmay be evenly divided between the device and a paired device (e.g.smartphone), with only a small amount of delegation to a server of acloud-based service. According to a second delegation model 12720 (whichmay, e.g., be applicable for devices manufactured in a three-yeartimeframe), most processing may be handled locally by the device (e.g.due to predicted advances in memory, CPU, and energy capacity in a smallform factor). In this second model, some processing may be delegated toa server (e.g., more than in the first delegation model, due to improvednetwork connectivity) and only a small amount of delegation may occur tothe locally paired device. According to a third delegation model 12730(which may, e.g., be applicable for devices manufactured in a five-yeartimeframe), all or almost all processing tasks may be evenly dividedbetween the device and a server of a cloud-based service, with no oralmost no processing being delegated to a locally-paired device. Anynumber of delegation models may be created, as the factors taken intoaccount by a delegation model are dynamic. As an example, all or almostall tasks may be performed locally on the device according to onedelegation model, and all or almost all tasks may be delegated by thedevice in another delegation model.

The device may choose to delegate functionality to a pairedprocessing-rich device (e.g. phone, computer, tablet, television,set-top box, refrigerator, washer, or dryer) or to the Internet based onthe energy reserves or connectivity bandwidth to each of theselocations. For example, a device with a powerful processor may delegateto the paired device when low on energy, or it may choose to delegate tothe Internet service when the paired device does not have sufficientpower reserves. Likewise, the system of the device may choose to processlocally if the connection to the Internet is showing higher latency toreduce the size of the data transfer.

In particular embodiments, an entire application or a portion of anapplication may be delegated by a user of the device to a paired deviceor vice versa. This may occur on a per-application basis. When theapplication on a target device (e.g. a television) is to be delegated tothe device, the target device may send a request over the pairedconnection (possibly via an intermediary device, such as a smartphone orpersonal computer) to load the application on the device. The device maythen act as a client to a server running on the paired device (e.g.television). Similarly, an application running on the device may bedelegated to the paired device (e.g. a video playing on the device maybe delegated to playing on a paired television). For example, if thedevice is running a first application, and a user of the device wants tointeract with a second application, the device may automaticallydelegate a task of the first application to be processed by anotherdevice (e.g. a paired television).

FIG. 128 illustrates an example of a decision flow in the deviceoperating according to a delegation model. In this example, animage-capture application is running on the device. A scene is capturedon the device 12810, and the device determines 12820 if it hassufficient CPU capacity for image feature calculations. If the devicedoes have enough CPU capacity, it calculates the features of interest inthe scene locally 12830. If the device does not have sufficient CPUcapacity, it may first determine 12840 if it is paired communicativelywith another device with more processing capability (e.g. a mobile phoneor a personal computer). If it is paired with such a device, the devicemay send data to the paired device so the paired device may calculatefeatures of interest in the image 12850. If the device is not pairedwith such a device, it may determine if it is connected to anInternet-based (e.g. cloud) service 12860. If not, the device performsno further action. If so, the device may send data to the cloud serviceso the service may calculate features of interest in the scene 12870.Features of interest may be calculated (wherever they are calculated)using any suitable algorithm including, for example, SURF. In thisexample, the features of interest may be compared to a local catalog oran Internet-based service to determine whether any matches are found(and if so, relevant information of interest) 12880. If a match is found12890, the result may be presented to a user on the device 12895. If nomatch is found, no further action is taken.

In particular embodiments, a camera or other optical sensor of thedevice may be used to recognize any gestures performed by the user (e.g.in the space between the camera and a target in the real world). Thesegestures may, for example, be used to act upon the data presented (e.g.the real world target, such as a sign including text) or may be used topoint to particular items upon which augmented reality functions may beperformed. For example, the user may point to a word on a sign, causingthe device to translate it and display the translation to the user. FIG.17 illustrates two examples of images captured by a camera of thedevice. In one example, a truck 1725 and the hand 1720 of a user of thedevice are both within the angle of view of a camera 1705 of the deviceand displayed by the device (shown at 1710). As such, gestures performedby the user upon the truck may be recognized by the device and processedby device to provide, for example, AR functionality. In the secondexample, only the truck is within the angle of view of the camera (shownat 1715), and as such, gestures performed by the user are not capturedor recognized by the device. Gesture recognition may also be delegatedby the device.

In particular embodiments, objects or images may be recognized by thedevice when they are within the frame of view of a camera of the device.As described herein, there may be multiple ways for the device torecognize an object. As one example, a gesture performed by the user(e.g. a pointing gesture indicating a particular object) may enable ARfunctionality on the device and cause the device to recognize theobject. As another example, automatic object recognition may occur when,for example, the user positions the camera for a certain amount of timeon a particular object (e.g. a section of text). As a third example,object recognition or AR functionality may be enabled explicitly by theuser when, for example, the user taps or touches the display (or, e.g.,clicks the outer ring) when the camera of the device has captured anobject of interest. Global object recognition may, in some instances, becomputationally intensive and error-prone. As such, in particularembodiments, a limiting set (e.g. the pages of a magazine or catalog ora catalog of a particular type of object such as plant leaves or bookcovers) may be applied to improve accuracy. There exist a number ofchoices for calculation of feature vectors from images, which thedesigner of the system for the device may select from. In someinstances, the conversion of feature vectors between differentapproaches may be computationally expensive, so that the choice of thedatabase of possible matches is replicated on the device. Thecalculation of feature vectors may be delegated, as described herein.

In particular embodiments, barcodes of various types may be recognizedby the device. These barcodes may be used to query Internet-basedservices for additional data, as well as options to purchase, review, orbookmark the barcoded item for future review. While two-dimensionalbarcodes may generally be read directly, the system of the device mayoffer an addition close-focus mode for particularly small orone-dimensional barcodes to improve recognition rate. Should the systemlack the ability to decode the barcode, it may simply focus the camera,take a picture, and delegate recognition to a remote service, asdescribed herein. FIGS. 129A-129D illustrate an example of barcoderecognition mode. The device may be pointed at an item (129A), recognizethe item (129B), display additional information obtained from theInternet about the item (129C), and provide the user an interface topurchase the item (129D).

In particular embodiments, the device may perform translation.Translation functionality may be divided into two portions: opticalcharacter recognition (OCR), and translation of recognized characters,words, or phrases. OCR may be completed on the device or delegated (e.g.to a paired processing device) to reduce the amount of data to betranslated by the device. Simple word translations may be performed onthe device or delegated (e.g. to a paired processing device). As withother functionality described herein, part or all of the recognition ortranslation process may be delegated as needed. The user may optionallyuse a gesture to indicate the word to be translated, as shown in FIG.130 (e.g. the word “Warning”). Since individual words may becircumscribed by white space, the system may segment the word beforeattempting translation. Additionally, if the device can perform OCR withlow latency, it may show the text to the user so that the user knowswhen the device is targeting and correctly recognizing the correct text.If automatic OCR is enabled, then the device may automatically identifyimages in the angle of view of an outward-facing camera and present onthe device display information about the identified images. If automatictranslation is enabled, then the device may automatically translate textin the angle of view of the outward-facing camera and present thetranslated text on the device display.

FIGS. 131A-131D illustrate examples of the device operating in variousaugmented reality modes described herein, including barcode recognitionmode (131A), image recognition mode (131B), OCR and translate mode(131C), and object recognition mode (131D).

FIG. 132 illustrates an example of the overall flow of actions for anaugmented reality system for the device. Although this exampleillustrates an image capture application, any suitable task or processon the device may follow a similar flow. Additionally, any task afterthe device captures an image and before the device displays results tothe user may (as suitable) be delegable by the device. In this example,an image from a camera of the device is captured (in the image capturesection 13210), pre-processed (in section 13220), features are extractedand recognized to produce image recognition results (in section 13230),and any objects may be recognized (in section 13240). Object data may beformatted for action by a user of the device. The user may activate theaugmented reality mode of the device 13211 (e.g. via a user gesture orpointing the camera of the device at an object for a pre-determinedamount of time), and an image in the view of the camera 13212 may becaptured (e.g. based on a trigger event such as a user input orautomatic camera activation) by device camera 13213 to produce a cameraimage 13214. At this point, the pre-processing stage 13220 may beentered. Pre-processing 13220 may, for example, include contrastenhancement, grayscale conversion, sharpening, or down-sampling. Inparticular embodiments, the camera may operate in a general augmentedreality mode in which anything in front of the camera may be processedand recognized. In other embodiments, the camera may operate in specificmodes (e.g. OCR, barcode, or visual marker) and recognize onlyparticular items when in such a mode. In particular embodiments, if itis determined that the image may include known shapes, symbols, ororganizations of shapes or symbols (e.g. if the camera or device is inOCR mode, barcode mode, or visual marker mode), AR image processing mayproceed on a first path. This first path begins with preliminaryprocessing 13221, proceeds to segmentation 13231 (which may, forexample, determine symbol or symbol group boundaries such as letters orwords), and commences with one or more of optical character recognition13234 (e.g. if it is determined the image may contain characters,determining what those characters are), barcode recognition 13235 (e.g.if it is determined the image may contain a barcode, recognizing thebarcode), or visual marker recognition (e.g. recognizing other types ofvisual markers) 13236 (e.g. for all other types of visual markers). Theresults of this first path are sent to object recognizer 13242. Inparticular embodiments, if it is determined that the image may includefeatures that are not necessarily known, AR image processing may proceedon a second path. The second path begins with feature extraction 13222(e.g. in which the presence of edges or lines, changes in angles oflines, edges, points of interest, or patterns, are detected in thecaptured image). The second path proceeds to image recognition 13232, inwhich the features of the image are compared with feature data from arecognition database 13233 (which may, for example, reside on thedevice, on a locally-paired device, or on a remote server or computer).The results of the image recognition comparison are provided 13237 andsent to the object recognizer 13242. In the object recognition section13240, the first and second paths converge at the object recognizer13242. Here, results from an object database 13241 are used to recognizeobjects (e.g. that a phone recognized using the image recognitiondatabase 13233 is a particular brand and model of phone). Object data13243 about the object recognized by recognizer 13242 (e.g. the price ofthe model of phone recognized, or where the phone may be available forpurchase) may be provided. For text, there may be definitions ortranslations that occur and are displayed to the user. For barcodes,there may be product information and links to buy the recognized objectthat are displayed to the user. In particular embodiments, the data maybe purely descriptive (e.g. the price of the phone) or may be active(e.g. a link where the user may purchase the phone). If the dataincludes action data 13244, then an action controller 13250 (whichcontrols, formats, and outputs a GUI for the user of the device) mayshow a UI to the user 13255 including the active data (e.g. the link forpurchasing the phone). If the user selects an action 13260 (e.g.clicking the link), then the action controller shows the action UI tothe user 13265 (e.g. opening of the link), and if the action isconfirmed 13270, then the action (e.g. the actual opening of the webpageassociated with the link) is performed 13275.

FIG. 133 illustrates an example of a network environment. As describedherein, in particular embodiments, the device 13310 may be paired withother devices (e.g. nearby devices). The device may connect directly toa personal area network 13320 (which may bridge via other devices on thesame network to a local area network), or the device may connect to alocal area network 13330 directly. The personal area network mayinclude, for example, non-WI-FI radio technology, such as BLUETOOTH,NFC, or ZIGBEE. The personal area network may, for example, include asmart media gateway 13322 (e.g. a media server), a smart TV 13324,another processing provider 13326, or a phone 13328. Phone 13328 mayallow the device to connect to a cellular network 13340, and from thereto the Internet 13350. The local area network 13330 may include, forexample, WI-FI with or without authentication. The local area networkmay, for example, include a local wireless network router 13332, smartmedia devices 13334, smart appliances 13336, and home automationtechnology 13338. The local area network may, in turn, connect to theglobal Internet 13350 via, for example, local router 13332 that connectsto an Internet service (e.g. a proprietary cloud service 13352 or othercloud service partners 13354). Some devices may be reached by the deviceeither via direct access (e.g. through the personal area network) orthrough the local area network. Those devices reachable by the devicemay be paired with the device and may be controlled by the device orcontrol the device. The device may connect to the personal area networkor the local area network using, for example, any suitable RFtechnology. As shown in FIG. 133, pairing to a target device in theperiphery may first occur over the RF network. This allows the device toknow what is “nearby”. This may happen over the personal area network(e.g. an ad-hoc or peer-to-peer network), or may use a mediated networksuch as 802.11 wireless (e.g. the local area network). Once aneighborhood is established, the device may request that nearby devicesenter pairing mode. This may be done either directly or via a pairedprocessing device with a greater gamut of connectivity options, such asa mobile phone. Once the target devices have entered pairing mode, theymay exhibit their pairing signals. For example, devices with displaysmay show a visual tag on their display, while others may enable an NFCtag allowing for a scanner to identify them. Other approaches such asselection from a list or by pin code may also be used. Once a device isuniquely identified as a pairing target, the device may exchange asecurity token with the target device to finalize the pairing.

FIG. 134 illustrates an example of different types of pairing technologythat may be used to pair a target device with the device. The targetdevice, which may be a smart device such as a phone, may include passiveNFC tags 13402 or active NFC transmitters 13404 (which may be recognizedby a NFC tag reader 13420 and NFC decoder 13428 of the device); an NFCdecoder 13406 (which may recognize NFC tags written by the NFC tagwriter 13422 of the device), passive visual tags 13408 (e.g. stickers),barcodes 13410, or other display information 13412 (which may berecognized by a camera 13424 of the device); or other pairing system13416. An active tag generator 13414 of the target device may create thedisplay information 13412 or provide information to the other pairingsystem 13416 of the target device (which is recognized by a mirrorpairing system 13426 with pairing code decoder 13438 of the device). Thedevice may write data to NFC tags (e.g. with an NFC tag writer 13422) totransmit this data to other target devices that may be paired to thedevice. Tags written by the device may be recognized by NFC tag decoders13406 on a target device. The device may include any of a number ofdecoders including barcode decoder 13430, visual tag decoder 13432,image recognizer 13434, or other image-based decoder 13436 (e.g. adecoder for QR codes, logos, or blink patterns of LEDs), all takinginput from camera 13424 of the device. After the device receives andrecognizes pairing information, it may decode (e.g. through a variety ofdecoders) the relevant information to proceed with pairing with thetarget device. In particular embodiments, pairing may be achieved usingmotion—a motion-sensitive target device (e.g. mobile phone or remote)may be paired with the device by holding and moving the target device inthe same hand as the device (e.g. if both devices includeaccelerometers, the similar pattern of motion may be detected and usedto pair the devices). As another example, a fixed target device may bepaired with the device by, for example, tapping the fixed target devicewith a random pattern while holding the fixed target device in the samehand as the device (e.g. if both devices include touch detection, thesimilar pattern of tapping may be detected and used to pair thedevices). Additionally, pairing may be done using audio—if the deviceand a target device both have audio reception capabilities, a user maymake a sound (e.g. say a phrase) that both devices detect and then setup a pairing. Any suitable technology (including, e.g., augmentedreality functions) of the device may be used to pair with and controllocal devices. The device and target device may each connect to otherpossible intermediary network devices 13440, and also to a local areanetwork 13450.

FIG. 135 illustrates an example process for pairing a target device(e.g. using any of the methods described herein) with the device. Oncepairing mode is enabled 13510, the device determines if the RF networkcontains pairable target devices 13512. If not, no further action istaken (e.g. the device may continue to scan periodically). If so, thedevice may request that the pairable devices enter pairing mode 13514.The device may then proceed (in any order, or in a parallel fashion) toscan, via different available technologies, for available targetdevices. These may include NFC tag scans 13516, visual tag scans in thecamera's angle of view 13518, barcode scans in the camera's angle ofview 13520, or any other method 13522. If a target device is detectedvia one of these methods, the target device is paired to the device13524. Once the pairing has occurred, the device may show menu items tothe user for controlling the paired device(s). The device may allow forboth visual and motion-based gestural control of the paired devices. Forexample, the user may gesture (e.g. wave her hand) to change channels ona paired television, or may make a pinching gesture to transfer videomedia from the device to a paired display (using, e.g., ARfunctionality). Device control mediated over an RF network may be bothlocal and securable. FIG. 136 illustrates example controls enabled onthe device for a paired and controlled television including an activeON/OFF icon 13610, favorite channels 13620, a current channel display13630, and volume 13640. As described herein, any suitable input fromthe user may be used to control functionality of a paired device. Forexample, gesture input, click or press input, or touch input may beused, for example, to change channels, adjust volume, or control otherfunctions of the paired television.

In particular embodiments, a pairing and control model for the devicemay include the following characteristics. The device may function asthe host for an application that interacts with or controls one or morefunctions of a remote device (e.g. an appcessory such as a controllablethermostat). A smartphone (or other locally-paired device), which mayhave previously been the host for the application, may now functionmerely as a local target device to which the device may delegate certainfunctions related to the interaction or control of the remote device(e.g. longer-range wireless connectivity to the remote device, sendingcommands to the remote device, receiving data from the remote device, orprocessing tasks). Control of the remote appcessory device may be doneby the device using any suitable means including, for example, visualmeans (e.g. using the camera) or motion-based gestures. In otherembodiments, the locally-paired smartphone may continue to function asthe host for the application that interacts with the remote appcessory,but the device may provide some or all of the user interface for datainput and output to and from the application (e.g. a “light” version ofthe application hosted by the smartphone). For example, the user maycontrol the application using the device, but the smartphone may stillfunction as the host of the application.

In particular embodiments, the device may be operable with one or moreservices. These services may fall in categories including security,energy, home automation and control, content sharing, healthcare, sportsand entertainment, commerce, vehicles, and social applications.

Example security applications include the following. The device mayauthenticate a user (who is wearing the unlocked device) to anotherdevice near the user (e.g. paired with the device). The device may beunlocked with a code entered by the user using any suitable inputincluding, for example, rotating the outer ring of the device. As anexample, while a user rotates (or presses or clicks) the outer ring, thedisplay may show alphanumeric or symbolic data corresponding to therotation (or press or click) by the user. If, for example, the userrotates the outer ring one rotational increment in a clockwise direction(or, e.g., clicks or presses the outer ring once), the display may showthe user a “1,” and if the user rotates the outer ring two rotationalincrements (e.g. within a certain period of time, such as a millisecond)in a clockwise direction (or, e.g., clicks or presses the outer ringtwice), the display may show the user a “2.” In particular embodiments,the display of alphanumeric or symbolic data corresponding to a rotation(or press or click) by the user may allow the user to unlock the deviceusing the metaphor of a combination lock. The device may also beunlocked using biometric data (e.g. by skin or bone signatures of theuser).

In an example energy application, the device may automatically displayinformation about the energy consumption of the room or other locationin which the user is located. The device may also be able to displayinformation about the energy consumption of other paired devices andupdate all of this information dynamically as the user changes location.

In an example home control application, the user may select and directlycontrol paired home-control devices using, for example, rotation of theouter ring or a gesture input.

The user may use gestures to control the sharing or transfer of contentto or from the device (e.g. transferring video playing on the device toa paired television, as described herein). Additionally, auxiliaryinformation (e.g. movie subtitles) may be provided on the device forcontent shown on another, larger device (e.g. television screen playingthe movie).

The device may automatically determine a healthcare context (e.g. if theuser is exercising or sleeping). When it determines this context, thedevice may open applications corresponding to the healthcare context(e.g. for recording heart rate during exercise, movement duringexercise, duration of exercise, pulse oximetry during exercise, sleeppatterns, duration of sleep, or galvanic skin response). The device may,for example, measure a user's health-related data (e.g. heart rate,movement, or pulse oximetry) and send some or all of this data to apaired device or a server. Although illustrated in the healthcarecontext, the determination of a relevant context (e.g. based on a user'sbehavior), opening of corresponding applications, recording of data, ortransmission of this data may be applicable in any suitable context.

The device may assist in sports-related applications such as, forexample, automatically assessing a golf swing of the user and suggestingcorrections.

In a commercial setting, the device may automatically identify a product(e.g. using RFID, NFC, barcode recognition, or object recognition) whenthe user picks up the product and may provide information about theproduct (e.g. nutrition information, source information, or reviews) orthe option to purchase the product. Payment for the product may, forexample, be accomplished using visual barcode technology on the device.In particular embodiments, the device may be used to pay for a productusing NFC, RFID, or any other suitable form of short-distancecommunication. During payment, the user's information may, for example,be authenticated by the device, which may detect the user's biometricinformation (e.g. bone structure or skin signature). The device may alsoautomatically provide an indication to the user (e.g. a vibration) whenthe user is near a product on her shopping list (e.g. stored in thedevice) or another list (e.g. a wish list of the user's friend).

The device may function as a key for unlocking or turning on one or morevehicles. The user may, for example, enter a code using the outer ringto unlock or turn on the vehicle (e.g. using NFC technology), asdescribed earlier. In particular embodiments, both user biometricinformation and a code entered by the user may be required to unlock thecar, allowing for enhanced security for a car-based application.Additionally, the device may include profiles for one or more users,each profile containing vehicle settings (e.g. temperature or seatposition). As another example, biometric information of a particularuser may be used not only to unlock the device, but also to determinewhich user profile to load during the car's operation. The proximity ofthe device to the vehicle may automatically cause the vehicle toimplement the vehicle settings of the profile of the user. The devicemay also be operable for GPS navigation (either directly on the deviceor when paired with and controlling a phone, for example).

The device may access and operate in conjunction with a service thatprovides support for mixed-reality games or massively multi-playerreality-based games. This functionality may, for example, includeregistration, management of user data (e.g. user profiles andgame-related data such as levels completed or inventories of supplies),and management of accomplishment lists. The functionality of the deviceand the service may also include management of connectivity (e.g.concentrator functionality) that handles fragile wireless communicationchannels and provides a unified API to third party game servers.

The device may access and operate in conjunction with a service thatallows a user of the device to publish locations, check-ins, or otherlocation-based data that allows various services to access a consistentreservoir of the most current information regarding the position andstatus of the user. As an example, the user of the device may findfriends using similar devices. The service and device together mayhandle status updates, profile management, application accesspermissions, blacklists, or user-to-user access permissions. The servicemay be a trusted and centralized touchpoint for private data. Bycombining access to a unified location service, energy and battery lifemay, in particular embodiments, be conserved. In particular embodiments,certain functionality tokens may be made available based on the positionof the user. An application may, for example, check on the device to seeif this token is available and act accordingly. On the server side, APIsmay allow developers to see use of the tokens or allow for redemption.In particular embodiments, information may be distributed by the deviceto other users (e.g. a single other user, or in broadcast mode tomultiple users).

The device may access and operate in conjunction with a service thatprovides a unified polling interface that allows devices to receive andsend polls. The device and service together may manage distributionlists, scoring criteria, and poll availability frames (both temporal andgeographic, for example). This service may be exposed on the device andon a server such that third parties may use APIs to write applicationsand receive results back via online APIs.

In particular embodiments, the device may access and operate inconjunction with a service that provides optimizations for thepresentation of text, images, or other information on a circular displayof the device. As an example, a web site may be rendered or formattedfor display on a computer monitor, but a service may customize therendering and formatting for a smaller, circular display by emphasizingimages and truncating text. The customized rendering and formatting may,for example, be a task delegable among the device and one or moreservers or locally-paired devices. This service may also include news oradvertising services.

FIG. 137 illustrates an example computer system 13700. In particularembodiments, one or more computer systems 13700 perform one or moresteps of one or more methods described or illustrated herein. Inparticular embodiments, one or more computer systems 13700 providefunctionality described or illustrated herein. In particularembodiments, software running on one or more computer systems 13700performs one or more steps of one or more methods described orillustrated herein or provides functionality described or illustratedherein. Particular embodiments include one or more portions of one ormore computer systems 13700. Herein, reference to a computer system mayencompass a computing device, and vice versa, where appropriate.Moreover, reference to a computer system may encompass one or morecomputer systems, where appropriate.

This disclosure contemplates any suitable number of computer systems13700. This disclosure contemplates computer system 13700 taking anysuitable physical form. As example and not by way of limitation,computer system 13700 may be an embedded computer system, asystem-on-chip (SOC), a single-board computer system (SBC) (such as, forexample, a computer-on-module (COM) or system-on-module (SOM)), adesktop computer system, a laptop or notebook computer system, aninteractive kiosk, a mainframe, a mesh of computer systems, a mobiletelephone, a personal digital assistant (PDA), a server, a tabletcomputer system, or a combination of two or more of these. Whereappropriate, computer system 13700 may include one or more computersystems 13700; be unitary or distributed; span multiple locations; spanmultiple machines; span multiple data centers; or reside in a cloud,which may include one or more cloud components in one or more networks.Where appropriate, one or more computer systems 13700 may performwithout substantial spatial or temporal limitation one or more steps ofone or more methods described or illustrated herein. As an example andnot by way of limitation, one or more computer systems 13700 may performin real time or in batch mode one or more steps of one or more methodsdescribed or illustrated herein. One or more computer systems 13700 mayperform at different times or at different locations one or more stepsof one or more methods described or illustrated herein, whereappropriate.

In particular embodiments, computer system 13700 includes a processor13702, memory 13704, storage 13706, an input/output (I/O) interface13708, a communication interface 13710, and a bus 13712. Although thisdisclosure describes and illustrates a particular computer system havinga particular number of particular components in a particulararrangement, this disclosure contemplates any suitable computer systemhaving any suitable number of any suitable components in any suitablearrangement.

In particular embodiments, processor 13702 includes hardware forexecuting instructions, such as those making up a computer program. Asan example and not by way of limitation, to execute instructions,processor 13702 may retrieve (or fetch) the instructions from aninternal register, an internal cache, memory 13704, or storage 13706;decode and execute them; and then write one or more results to aninternal register, an internal cache, memory 13704, or storage 13706. Inparticular embodiments, processor 13702 may include one or more internalcaches for data, instructions, or addresses. This disclosurecontemplates processor 13702 including any suitable number of anysuitable internal caches, where appropriate. As an example and not byway of limitation, processor 13702 may include one or more instructioncaches, one or more data caches, and one or more translation lookasidebuffers (TLBs). Instructions in the instruction caches may be copies ofinstructions in memory 13704 or storage 13706, and the instructioncaches may speed up retrieval of those instructions by processor 13702.Data in the data caches may be copies of data in memory 13704 or storage13706 for instructions executing at processor 13702 to operate on; theresults of previous instructions executed at processor 13702 for accessby subsequent instructions executing at processor 13702 or for writingto memory 13704 or storage 13706; or other suitable data. The datacaches may speed up read or write operations by processor 13702. TheTLBs may speed up virtual-address translation for processor 13702. Inparticular embodiments, processor 13702 may include one or more internalregisters for data, instructions, or addresses. This disclosurecontemplates processor 13702 including any suitable number of anysuitable internal registers, where appropriate. Where appropriate,processor 13702 may include one or more arithmetic logic units (ALUs);be a multi-core processor; or include one or more processors 13702.Although this disclosure describes and illustrates a particularprocessor, this disclosure contemplates any suitable processor.

In particular embodiments, memory 13704 includes main memory for storinginstructions for processor 13702 to execute or data for processor 13702to operate on. As an example and not by way of limitation, computersystem 13700 may load instructions from storage 13706 or another source(such as, for example, another computer system 13700) to memory 13704.Processor 13702 may then load the instructions from memory 13704 to aninternal register or internal cache. To execute the instructions,processor 13702 may retrieve the instructions from the internal registeror internal cache and decode them. During or after execution of theinstructions, processor 13702 may write one or more results (which maybe intermediate or final results) to the internal register or internalcache. Processor 13702 may then write one or more of those results tomemory 13704. In particular embodiments, processor 13702 executes onlyinstructions in one or more internal registers or internal caches or inmemory 13704 (as opposed to storage 13706 or elsewhere) and operatesonly on data in one or more internal registers or internal caches or inmemory 13704 (as opposed to storage 13706 or elsewhere). One or morememory buses (which may each include an address bus and a data bus) maycouple processor 13702 to memory 13704. Bus 13712 may include one ormore memory buses, as described below. In particular embodiments, one ormore memory management units (MMUs) reside between processor 13702 andmemory 13704 and facilitate accesses to memory 13704 requested byprocessor 13702. In particular embodiments, memory 13704 includes randomaccess memory (RAM). This RAM may be volatile memory, where appropriate,and this RAM may be dynamic RAM (DRAM) or static RAM (SRAM), whereappropriate. Moreover, where appropriate, this RAM may be single-portedor multi-ported RAM. This disclosure contemplates any suitable RAM.Memory 13704 may include one or more memories 13704, where appropriate.Although this disclosure describes and illustrates particular memory,this disclosure contemplates any suitable memory.

In particular embodiments, storage 13706 includes mass storage for dataor instructions. As an example and not by way of limitation, storage13706 may include a hard disk drive (HDD), a floppy disk drive, flashmemory, an optical disc, a magneto-optical disc, magnetic tape, or aUniversal Serial Bus (USB) drive or a combination of two or more ofthese. Storage 13706 may include removable or non-removable (or fixed)media, where appropriate. Storage 13706 may be internal or external tocomputer system 13700, where appropriate. In particular embodiments,storage 13706 is non-volatile, solid-state memory. In particularembodiments, storage 13706 includes read-only memory (ROM). Whereappropriate, this ROM may be mask-programmed ROM, programmable ROM(PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM),electrically alterable ROM (EAROM), or flash memory or a combination oftwo or more of these. This disclosure contemplates mass storage 13706taking any suitable physical form. Storage 13706 may include one or morestorage control units facilitating communication between processor 13702and storage 13706, where appropriate. Where appropriate, storage 13706may include one or more storages 13706. Although this disclosuredescribes and illustrates particular storage, this disclosurecontemplates any suitable storage.

In particular embodiments, I/O interface 13708 includes hardware,software, or both, providing one or more interfaces for communicationbetween computer system 13700 and one or more I/O devices. Computersystem 13700 may include one or more of these I/O devices, whereappropriate. One or more of these I/O devices may enable communicationbetween a person and computer system 13700. As an example and not by wayof limitation, an I/O device may include a keyboard, keypad, microphone,monitor, mouse, printer, scanner, speaker, still camera, stylus, tablet,touch screen, trackball, video camera, another suitable I/O device or acombination of two or more of these. An I/O device may include one ormore sensors. This disclosure contemplates any suitable I/O devices andany suitable I/O interfaces 13708 for them. Where appropriate, I/Ointerface 13708 may include one or more device or software driversenabling processor 13702 to drive one or more of these I/O devices. I/Ointerface 13708 may include one or more I/O interfaces 13708, whereappropriate. Although this disclosure describes and illustrates aparticular I/O interface, this disclosure contemplates any suitable I/Ointerface.

In particular embodiments, communication interface 13710 includeshardware, software, or both providing one or more interfaces forcommunication (such as, for example, packet-based communication) betweencomputer system 13700 and one or more other computer systems 13700 orone or more networks. As an example and not by way of limitation,communication interface 13710 may include a network interface controller(NIC) or network adapter for communicating with an Ethernet or otherwire-based network or a wireless NIC (WNIC) or wireless adapter forcommunicating with a wireless network, such as a WI-FI network. Thisdisclosure contemplates any suitable network and any suitablecommunication interface 13710 for it. As an example and not by way oflimitation, computer system 13700 may communicate with an ad hocnetwork, a personal area network (PAN), a local area network (LAN), awide area network (WAN), a metropolitan area network (MAN), body areanetwork (BAN), or one or more portions of the Internet or a combinationof two or more of these. One or more portions of one or more of thesenetworks may be wired or wireless. As an example, computer system 13700may communicate with a wireless PAN (WPAN) (such as, for example, aBLUETOOTH WPAN), a WI-FI network, a WI-MAX network, a cellular telephonenetwork (such as, for example, a Global System for Mobile Communications(GSM) network), or other suitable wireless network or a combination oftwo or more of these. Computer system 13700 may include any suitablecommunication interface 13710 for any of these networks, whereappropriate. Communication interface 13710 may include one or morecommunication interfaces 13710, where appropriate. Although thisdisclosure describes and illustrates a particular communicationinterface, this disclosure contemplates any suitable communicationinterface.

In particular embodiments, bus 13712 includes hardware, software, orboth coupling components of computer system 13700 to each other. As anexample and not by way of limitation, bus 13712 may include anAccelerated Graphics Port (AGP) or other graphics bus, an EnhancedIndustry Standard Architecture (EISA) bus, a front-side bus (FSB), aHYPERTRANSPORT (HT) interconnect, an Industry Standard Architecture(ISA) bus, an INFINIBAND interconnect, a low-pin-count (LPC) bus, amemory bus, a Micro Channel Architecture (MCA) bus, a PeripheralComponent Interconnect (PCI) bus, a PCI-Express (PCIe) bus, a serialadvanced technology attachment (SATA) bus, a Video Electronics StandardsAssociation local (VLB) bus, or another suitable bus or a combination oftwo or more of these. Bus 13712 may include one or more buses 13712,where appropriate. Although this disclosure describes and illustrates aparticular bus, this disclosure contemplates any suitable bus orinterconnect.

Herein, a computer-readable non-transitory storage medium or media mayinclude one or more semiconductor-based or other integrated circuits(ICs) (such, as for example, field-programmable gate arrays (FPGAs) orapplication-specific ICs (ASICs)), hard disk drives (HDDs), hybrid harddrives (HHDs), optical discs, optical disc drives (ODDs),magneto-optical discs, magneto-optical drives, floppy diskettes, floppydisk drives (FDDs), magnetic tapes, solid-state drives (SSDs),RAM-drives, SECURE DIGITAL cards or drives, any other suitablecomputer-readable non-transitory storage media, or any suitablecombination of two or more of these, where appropriate. Acomputer-readable non-transitory storage medium may be volatile,non-volatile, or a combination of volatile and non-volatile, whereappropriate.

Herein, “or” is inclusive and not exclusive, unless expressly indicatedotherwise or indicated otherwise by context. Therefore, herein, “A or B”means “A, B, or both,” unless expressly indicated otherwise or indicatedotherwise by context. Moreover, “and” is both joint and several, unlessexpressly indicated otherwise or indicated otherwise by context.Therefore, herein, “A and B” means “A and B, jointly or severally,”unless expressly indicated otherwise or indicated otherwise by context.

The scope of this disclosure encompasses all changes, substitutions,variations, alterations, and modifications to the example embodimentsdescribed or illustrated herein that a person having ordinary skill inthe art would comprehend. The scope of this disclosure is not limited tothe example embodiments described or illustrated herein. Moreover,although this disclosure describes and illustrates respectiveembodiments herein as including particular components, elements,feature, functions, operations, or steps, any of these embodiments mayinclude any combination or permutation of any of the components,elements, features, functions, operations, or steps described orillustrated anywhere herein that a person having ordinary skill in theart would comprehend. Furthermore, reference in the appended claims toan apparatus or system or a component of an apparatus or system beingadapted to, arranged to, capable of, configured to, enabled to, operableto, or operative to perform a particular function encompasses thatapparatus, system, component, whether or not it or that particularfunction is activated, turned on, or unlocked, as long as thatapparatus, system, or component is so adapted, arranged, capable,configured, enabled, operable, or operative.

While this disclosure describes particular structures, features,interactions, and functionality in the context of a wearable device,this disclosure contemplates that those structures, features,interactions, or functionality may be applied to, used for, or used inany other suitable electronic device (such as, for example, a smartphone, tablet, camera, or personal computing device), where appropriate.

What is claimed is:
 1. A method comprising, by a wearable computingdevice: displaying, at the wearable computing device, one or morefunctions associated with a remote device, wherein the remote device ispaired with the wearable computing device; receiving, at the wearablecomputing device, a user input from a user; determining, based on theuser input, one or more intended operations of one or more of thefunctions by the user; sending, to the remote device, instructions forexecuting the intended operations of the one or more of the functionsassociated with the remote device; and displaying, at the wearablecomputing device, execution results of the intended operations of theone or more of the functions associated with the remote device.
 2. Themethod of claim 1, wherein the wearable computing device comprises: adevice body comprising: one or more processors; a memory; atouch-sensitive display; a rotatable element; and a detector configuredto detecting rotation of the rotatable element; an optical camera; and abiometric sensor in or on a base of the device body; and a band coupledto the device body.
 3. The method of claim 1, further comprising:capturing a media file by an optical camera of the wearable computingdevice; transferring the captured media file to the remote device;receiving, from the remote device, features of interest associated withthe captured media file determined by the remote device; identifyinginformation associated with the media file based on the features ofinterest; and displaying the identified information at the wearablecomputing device.
 4. The method of claim 1, wherein: the remote deviceis paired with the wearable computing device using one or more of thefollowing: a BLUETOOTH connection between the remote device and thewearable computing device; a near-field communication (NFC) connectionbetween the remote device and the wearable computing device; or a WI-FIconnection between the remote device and the wearable computing device.5. The method of claim 1, wherein the remote device comprises one ormore of an appliance, a display device, a mobile device, an audiosystem, a video system, an automobile, or a personal computing device.6. The method of claim 1, further comprising: displaying content at thewearable computing device; and sending all or a portion of the contentto the remote device.
 7. The method of claim 1, further comprising:accessing content from the remote device; and displaying the accessedcontent at the wearable computing device.
 8. A wearable computing devicecomprising: one or more non-transitory computer-readable storage mediaincluding instructions; and one or more processors coupled to thestorage media, the one or more processors configured to execute theinstructions to: display, at the wearable computing device, one or morefunctions associated with a remote device, wherein the remote device ispaired with the wearable computing device; receive, at the wearablecomputing device, a user input from a user; determine, based on the userinput, one or more intended operations of one or more of the functionsby the user; send, to the remote device, instructions for executing theintended operations of the one or more of the functions associated withthe remote device; and display, at the wearable computing device,execution results of the intended operations of the one or more of thefunctions associated with the remote device.
 9. The wearable computingdevice of claim 8, wherein the wearable computing device furthercomprises: a device body comprising: one or more processors; a memory; atouch-sensitive display; a rotatable element; and a detector configuredto detecting rotation of the rotatable element; an optical camera; and abiometric sensor in or on a base of the device body; and a band coupledto the device body.
 10. The wearable computing device of claim 8,wherein the processors are further configured to execute theinstructions to: capture a media file by an optical camera of thewearable computing device; transfer the captured media file to theremote device; receive, from the remote device, features of interestassociated with the captured media file determined by the remote device;identify information associated with the media file based on thefeatures of interest; and display the identified information at thewearable computing device.
 11. The wearable computing device of claim 8,wherein: the remote device is paired with the wearable computing deviceusing one or more of the following: a BLUETOOTH connection between theremote device and the wearable computing device; a near-fieldcommunication (NFC) connection between the remote device and thewearable computing device; or a WI-FI connection between the remotedevice and the wearable computing device.
 12. The wearable computingdevice of claim 8, wherein the remote device comprises one or more of anappliance, a display device, a mobile device, an audio system, a videosystem, an automobile, or a personal computing device.
 13. The wearablecomputing device of claim 8, wherein the processors are furtherconfigured to execute the instructions to: display content at thewearable computing device; and send all or a portion of the content tothe remote device.
 14. The wearable computing device of claim 8, whereinthe processors are further configured to execute the instructions to:access content from the remote device; and display the accessed contentat the wearable computing device.
 15. A computer-readable non-transitorystorage media comprising instructions executable by a processor to:display, at a wearable computing device, one or more functionsassociated with a remote device, wherein the remote device is pairedwith the wearable computing device; receive, at the wearable computingdevice, a user input from a user; determine, based on the user input,one or more intended operations of one or more of the functions by theuser; send, to the remote device, instructions for executing theintended operations of the one or more of the functions associated withthe remote device; and display, at the wearable computing device,execution results of the intended operations of the one or more of thefunctions associated with the remote device.
 16. The media of claim 15,wherein the wearable computing device comprises: a device bodycomprising: one or more processors; a memory; a touch-sensitive display;a rotatable element; and a detector configured to detecting rotation ofthe rotatable element; an optical camera; and a biometric sensor in oron a base of the device body; and a band coupled to the device body. 17.The media of claim 15, wherein the instructions are further executableby the processor to: capture a media file by an optical camera of thewearable computing device; transfer the captured media file to theremote device; receive, from the remote device, features of interestassociated with the captured media file determined by the remote device;identify information associated with the media file based on thefeatures of interest; and display the identified information at thewearable computing device.
 18. The media of claim 15, wherein: theremote device is paired with the wearable computing device using one ormore of the following: a BLUETOOTH connection between the remote deviceand the wearable computing device; a near-field communication (NFC)connection between the remote device and the wearable computing device;or a WI-FI connection between the remote device and the wearablecomputing device.
 19. The media of claim 15, wherein the remote devicecomprises one or more of an appliance, a display device, a mobiledevice, an audio system, a video system, an automobile, or a personalcomputing device.
 20. The media of claim 15, wherein the instructionsare further executable by the processor to: display content at thewearable computing device; and send all or a portion of the content tothe remote device.